1
00:00:00.480 --> 00:00:01.140
Mick Coady: Meeting.
2
00:00:02.879 --> 00:00:03.780
Mick Coady: going.
3
00:00:09.599 --> 00:00:13.500
Mick Coady: Alright let's roll thanks everybody for showing up.
4
00:00:16.020 --> 00:00:25.740
Mick Coady: And really not much good feel for how long this will last a lot will depend on what kind of good conversation we have near.
5
00:00:26.430 --> 00:00:43.350
Mick Coady: near the end of the meeting on the agenda so without that that will just get going so here's what topics, I want to cover today, of course, go over the maybe the last time we'll talk about HP SS evacuation.
6
00:00:44.580 --> 00:00:56.730
Mick Coady: update on campaign storage and scratch uch usual and then update on the augmentation for campaign storage and then I think hopefully the will have a.
7
00:00:57.840 --> 00:01:07.410
Mick Coady: Good conversation about campaign stories and scratch usage, particularly on Rachel and NWS see after cheyenne is gone.
8
00:01:11.550 --> 00:01:27.240
Mick Coady: Okay, so, as of today, we've got only 23 days left until he pss is decommissioned, been a long time coming, and feels like i'll be glad when it's over with and i'm bet you guys will be glad to.
9
00:01:28.290 --> 00:01:36.120
Mick Coady: hear me stop talking about they were getting the notices daily bulletins and stuff, so I think we're in pretty good shape still.
10
00:01:37.260 --> 00:01:40.650
Mick Coady: Very few requests for.
11
00:01:42.000 --> 00:01:57.120
Mick Coady: HP SS access to be restored for those users that had gotten cut off, we got one just one user by the week or so ago wanted extra bandwidth you more.
12
00:01:57.600 --> 00:02:09.060
Mick Coady: Simon concurrent that tasks allowed so she could get some data off and that's going well gary's been busy looks like you're of late, yes, getting stuff off so.
13
00:02:10.530 --> 00:02:26.400
Mick Coady: no and no complaints or Christ of like you know stop we can't, we have to keep it going so it still will be cut off on October 1 and we'll move on.
14
00:02:28.110 --> 00:02:32.910
Mick Coady: Any of course interrupt me with any questions you might have one more here.
15
00:02:35.400 --> 00:02:43.170
Mick Coady: So on campaign storage utilization I think it's going pretty well, you can see the usage graph.
16
00:02:44.370 --> 00:02:49.770
Mick Coady: Hopefully, you can see my cursor on here you notice this kind of some sharp.
17
00:02:50.910 --> 00:02:55.710
Mick Coady: discontinuities here in the in the charts in the in the trends.
18
00:02:56.790 --> 00:03:06.870
Mick Coady: But in early August joey found an issue with the way he was calculating some some numbers in adjusted for that.
19
00:03:07.950 --> 00:03:08.640
Mick Coady: and
20
00:03:10.080 --> 00:03:14.490
Mick Coady: So the number of petabytes you see a pretty sharp increase in that.
21
00:03:15.630 --> 00:03:24.300
Mick Coady: fairly you know fairly modest increase or significant noticeable increase in the number of files I don't know what to make of that.
22
00:03:25.170 --> 00:03:38.100
Mick Coady: The number of petabytes allocated I don't know if that was just another adjustment that joey made or not, if we look at the petabytes used here i'm going to guess that this.
23
00:03:39.270 --> 00:03:39.870
Mick Coady: Nice.
24
00:03:41.340 --> 00:03:53.070
Mick Coady: trend line that we had from starting in January up through early August i'm going to guess that that could have been easily was supposed to have just been shifted up.
25
00:03:54.390 --> 00:03:56.070
Mick Coady: It just looks pretty.
26
00:03:57.750 --> 00:04:00.840
Mick Coady: Pretty consistent with the trend we're seeing since then so.
27
00:04:02.610 --> 00:04:05.370
Mick Coady: Again we're we're we're in good shape for now.
28
00:04:06.750 --> 00:04:12.330
Mick Coady: And I rely on on all of you to let me know or sizzle know if.
29
00:04:13.380 --> 00:04:15.240
Mick Coady: If everyone is.
30
00:04:18.210 --> 00:04:21.000
Mick Coady: It has enough we're if you're running in Danny.
31
00:04:22.020 --> 00:04:31.410
Mick Coady: squeeze on the amount of space you've got yeah okay thanks Gary I just saw your note about HP SSI since that's good.
32
00:04:32.880 --> 00:04:35.220
Mick Coady: Hopefully, most everyone else's too.
33
00:04:37.020 --> 00:04:47.130
Mick Coady: let's see and then and i'm going to ask you to kind of keep we can refer back to them, of course, but when we get into the conversation here in a few minutes try and keep these.
34
00:04:48.960 --> 00:04:51.780
Mick Coady: These numbers these you know these trends in mind.
35
00:04:53.040 --> 00:04:59.160
Mick Coady: scratch is it's consistently inconsistent, as you can see.
36
00:05:00.630 --> 00:05:08.250
Mick Coady: it's you know get get a lot of good usage, but we're in very good shape, this is on court blades scratch space today.
37
00:05:09.450 --> 00:05:11.610
Mick Coady: we've got good room.
38
00:05:11.850 --> 00:05:12.930
Mick Coady: And we'll.
39
00:05:13.710 --> 00:05:30.000
Mick Coady: I don't think anybody is concerned that we're going to have any issues before blade is decommission probably in late next year, not sure what's happened to the time frame on that given day rachel's delay so.
40
00:05:31.800 --> 00:05:35.040
Mick Coady: Maybe maybe we should I didn't think did mention that.
41
00:05:36.090 --> 00:05:52.740
Mick Coady: That it looks HP is being very non committal, right now, but the as we've mentioned before the delivery of direct show has been impacted by the worldwide shortage in semiconductors in the supply chain there.
42
00:05:54.000 --> 00:06:06.390
Mick Coady: That is it hasn't gotten any better, as of late so we're still waiting for something a little bit more definitive from HP in the contract to be revised to account for.
43
00:06:07.800 --> 00:06:17.100
Mick Coady: Any for further for the delay and we'll do our best to keep all users apprised as as the information firms up, and we know more.
44
00:06:19.200 --> 00:06:24.900
Mick Coady: All right, um if there are any questions about anything i've covered so far i'll move on.
45
00:06:26.190 --> 00:06:28.830
Mick Coady: Does anybody have any anything they want to talk about.
46
00:06:33.240 --> 00:06:35.130
Davide Del Vento: Maybe I haven't just a simple question.
47
00:06:35.550 --> 00:06:35.790
sure.
48
00:06:38.520 --> 00:06:40.140
Davide Del Vento: So glade is a.
49
00:06:41.850 --> 00:06:44.220
Davide Del Vento: Spectrum scale so it's IBM right.
50
00:06:46.050 --> 00:06:47.010
Mick Coady: it's.
51
00:06:48.540 --> 00:06:50.220
Davide Del Vento: as fast as the hardware.
52
00:06:50.820 --> 00:06:52.470
Davide Del Vento: Yes, yes yeah.
53
00:06:53.220 --> 00:06:58.860
Davide Del Vento: But the question is maybe, for you know for cheyenne it's the same vendor you know store HP.
54
00:06:59.370 --> 00:07:06.990
Davide Del Vento: So we can get an extension of support or something, but how about a bleeders say that if the ratio is delayed for a longer time.
55
00:07:07.560 --> 00:07:20.940
Davide Del Vento: Can is there an option to have a more you know not not having our contract without populate expire these data into the talk with somebody I don't know, maybe it's a question for fun more than you but.
56
00:07:22.200 --> 00:07:27.390
Mick Coady: it's a great question in yes or phone is actively pursuing that with dd in.
57
00:07:30.630 --> 00:07:33.390
Mick Coady: for contingencies right in case that.
58
00:07:34.410 --> 00:07:44.160
Mick Coady: The maintenance needs to be extended at this point, it probably will it's to some to some degree, but we don't know how much yet, but that.
59
00:07:45.510 --> 00:07:49.200
Mick Coady: That stone is not being left unturned okay.
60
00:07:50.250 --> 00:07:50.610
Davide Del Vento: Great.
61
00:07:50.670 --> 00:07:51.810
Mick Coady: Thank you yeah.
62
00:07:51.960 --> 00:07:52.680
Davide Del Vento: Good question.
63
00:07:55.140 --> 00:07:55.890
Mick Coady: anybody else.
64
00:07:59.520 --> 00:08:04.950
Mick Coady: Okay Susan getting into it like I mentioned in the email today.
65
00:08:05.910 --> 00:08:20.190
Mick Coady: Well i'm sorry Nick There is one last thing I did want to talk about the campaign storage augmentation i've mentioned before, a couple times that you know about the funding was approved and the delivery was expected last month in those all you know.
66
00:08:21.450 --> 00:08:26.520
Mick Coady: went as planned, that the hardware has been delivered in is up at NWS see.
67
00:08:29.580 --> 00:08:33.960
Mick Coady: Mostly joey is, of course, is working on the.
68
00:08:34.980 --> 00:08:46.800
Mick Coady: Along with other help from bill the hs G team is working to get it configured and get the software stacks updated so that it can.
69
00:08:47.880 --> 00:08:49.410
Mick Coady: successfully be deployed.
70
00:08:50.640 --> 00:09:01.140
Mick Coady: A couple of gotchas along the way, of course, but everything nothing that seems to suggest that anything will be delayed, and we still expect the.
71
00:09:02.160 --> 00:09:08.520
Mick Coady: new hardware and the extra capacity to be available sometime this fall.
72
00:09:10.170 --> 00:09:11.070
Mick Coady: Again, the.
73
00:09:12.420 --> 00:09:25.020
Mick Coady: mentioned before, that it's going to increase the campaign storage capacity to about 90 petabytes is what we expect and lab allocations will be increased.
74
00:09:25.770 --> 00:09:43.950
Mick Coady: Dave Hart will be in charge of that, but I was disabused of the notion that all of that 25 extra 25 petabytes will be allocated Dave is going to keep some in reserve for later.
75
00:09:45.000 --> 00:09:50.820
Mick Coady: How much I don't know he hasn't decided yet he's still working the numbers and wants to.
76
00:09:51.990 --> 00:09:53.550
Mick Coady: Put a finer pencil on that.
77
00:09:54.570 --> 00:09:57.210
Mick Coady: So I apologize if I missed lead.
78
00:09:58.410 --> 00:10:07.920
Mick Coady: For misleading or given false information in previous meetings, there will be an increase, but it won't be by the full 25 petabytes.
79
00:10:09.180 --> 00:10:11.160
Mick Coady: When it when the system.
80
00:10:12.450 --> 00:10:15.780
Mick Coady: gets expanded in a couple of months so.
81
00:10:18.570 --> 00:10:26.910
Mick Coady: yeah that's that's really all I have to say on that should have more firm information, of course, this time next month.
82
00:10:30.000 --> 00:10:30.480
Mick Coady: So.
83
00:10:32.220 --> 00:10:34.080
Mick Coady: Since we've met.
84
00:10:35.130 --> 00:10:45.540
Mick Coady: Back in August there's been some conversation going on, but with sizzle and some of the user communities about some users about.
85
00:10:47.370 --> 00:11:05.190
Mick Coady: camping storage and scratch usage on window Rachel comes online and after, particularly after blade is decommission so just kind of wanted to toss out some reminders of things that we've talked about before and.
86
00:11:06.480 --> 00:11:11.010
Mick Coady: Some pieces of data pieces of information that we can use for.
87
00:11:12.570 --> 00:11:18.930
Mick Coady: What I hope is a good conversation that follows and i've got a slide after that, after this with some questions so.
88
00:11:20.310 --> 00:11:30.510
Mick Coady: we're currently is the graph you can see, I didn't have the numbers these particular numbers on there, but we're at about 73% of.
89
00:11:31.860 --> 00:11:35.430
Mick Coady: Current capacity of of campaign storage.
90
00:11:36.570 --> 00:11:41.700
Mick Coady: which amounts to about 80% of the allocated what's been allocated.
91
00:11:43.530 --> 00:11:43.860
Mick Coady: What.
92
00:11:45.180 --> 00:12:06.840
Mick Coady: We haven't talked about before and it wasn't clear visit really well known to me or to some other folks until just a few weeks ago that keeping storage is not going to continue to grow indefinitely and infinitely right that the we've been operating on a pretty good clip here.
93
00:12:07.920 --> 00:12:21.210
Mick Coady: Since it came online of adding like 20 petabytes of the clip once or twice a year for the last three years, but are fun, let me know that.
94
00:12:22.350 --> 00:12:25.050
Mick Coady: The capacity will start plateauing.
95
00:12:26.160 --> 00:12:38.730
Mick Coady: sometime in 2223 and it'll depending on the current technologies and capacities that's going to be in the area of about 120 petabytes.
96
00:12:40.110 --> 00:12:50.460
Mick Coady: And at that point, the older discs will start being retired when new discs start coming on.
97
00:12:51.810 --> 00:12:52.770
Mick Coady: Hence the.
98
00:12:56.250 --> 00:13:04.740
Mick Coady: The ceiling or the plateau of around what we think right now will be about 120 petabytes.
99
00:13:06.780 --> 00:13:30.240
Mick Coady: So, and then today glade scratch it's been 15 petabytes since blade two came online we're at about a healthy 80% of that is being used, if you look back, we can flip back to the graph I showed a few minutes ago and occur in the current hundred and 20 day purge.
100
00:13:31.980 --> 00:13:32.400
Mick Coady: Now.
101
00:13:33.450 --> 00:13:37.200
Mick Coady: Campaign storage in the new scratch, maybe should have.
102
00:13:38.280 --> 00:13:49.140
Mick Coady: made that a little more clear in the news, the news grass, it will come within wc three will be mounted on both casper and Rachel unlike scratch today which.
103
00:13:50.100 --> 00:14:01.950
Mick Coady: i'm sorry which campaign storage is not mounted on cheyenne with it will be available from the Rachel and I know that there's been some talk here in this meeting that the.
104
00:14:04.170 --> 00:14:16.620
Mick Coady: You many of you expect campaign storage usage to maybe even take off, maybe in a hockey stick kind of trend once that comes online on Rachel.
105
00:14:18.900 --> 00:14:29.430
Mick Coady: And to remind you that in wc three scratch space will be 60 petabytes so rough you know pretty much four times what we currently enjoy the day.
106
00:14:30.720 --> 00:14:36.840
Mick Coady: And the current plan is to have a 190 day purge bossier approximately.
107
00:14:36.870 --> 00:14:37.800
Mick Coady: A half a year.
108
00:14:39.630 --> 00:14:42.390
Mick Coady: So the quite the.
109
00:14:43.410 --> 00:14:46.470
Mick Coady: August, so far as to call it a concern in.
110
00:14:47.730 --> 00:14:51.570
Mick Coady: Within that we wanted to address is.
111
00:14:52.800 --> 00:14:58.620
Mick Coady: Please don't laugh, the question about Weiss camping storage so popular it's think of it rhetorical.
112
00:15:00.510 --> 00:15:17.400
Mick Coady: And we'll do we do believe that camping stories will continue to be just generally favored over scratch space, why is that is it simply a matter of perch policies that we have in place Is there something else.
113
00:15:18.750 --> 00:15:33.090
Mick Coady: Is there something sizzle can or should do to encourage scratch usage on the Rachel within take advantage of that large and rather expensive space that will be available.
114
00:15:35.220 --> 00:15:45.480
Mick Coady: Would would you would the users benefit from project based scratch space and, in addition to not in lieu of but in addition to.
115
00:15:46.590 --> 00:15:49.500
Mick Coady: Personal scratch space personal user space.
116
00:15:50.760 --> 00:15:58.350
Mick Coady: And, are there any other questions that i'm not thinking to ask so with that i'm.
117
00:15:59.610 --> 00:16:04.320
Mick Coady: going to shut up and let you listen to what you guys have to say.
118
00:16:18.570 --> 00:16:19.680
Mick Coady: Nothing.
119
00:16:20.910 --> 00:16:23.850
Davide Del Vento: Well, there are a couple folks with arrays hand.
120
00:16:24.660 --> 00:16:27.960
Mick Coady: Okay i'm going to I can't see those.
121
00:16:28.410 --> 00:16:30.840
Davide Del Vento: Are Gary Gary was the first, so I think.
122
00:16:30.840 --> 00:16:31.260
Mick Coady: Thanks.
123
00:16:31.860 --> 00:16:33.090
Mick Coady: Okay Gary can you go them.
124
00:16:33.480 --> 00:16:36.600
Gary Strand: Well, yes, why you see as popular because it's not perch.
125
00:16:37.860 --> 00:16:42.120
Mick Coady: Well that's was pretty much my pretty much My guess but.
126
00:16:42.240 --> 00:16:52.350
Gary Strand: right because, because it makes you think about tedious details like how much space in my using where is it with files are they what data is that in.
127
00:16:53.280 --> 00:17:01.380
Gary Strand: People would much rather think about big picture things and not worry about the tedious details so that's quite a campaign storage is popular now.
128
00:17:04.890 --> 00:17:12.390
Gary Strand: My concern is that since we're using campaigns storage as a replacement for each process, which is part of its intent.
129
00:17:13.200 --> 00:17:25.230
Gary Strand: My concern is that campaign storage once the retro comes online campaign storage will fill up promptly, because people don't want to have to worry about Oh, is this data is this model right going to get purged or well.
130
00:17:26.340 --> 00:17:42.060
Gary Strand: And if campaign storage fills up that's bad because that would cause lots of things to crash and burn so my feeling is that you know goofy needs to be hardened and made more reliable so it's much more.
131
00:17:43.650 --> 00:17:49.470
Gary Strand: User usable for monitoring campaign storage and clean storage usage.
132
00:17:50.610 --> 00:18:03.330
Gary Strand: My old habit of getting like recursive listings and analyzing them does it work when you're approaching millions to you know millions and millions of files, so I think I mean goopy is a.
133
00:18:06.120 --> 00:18:21.900
Gary Strand: Great tool, but it just needs to be hardened and expanded, so it can be used more effectively and more efficiently by US Data managers, so that we can you know, monitor and check these thing or monitor and check these spaces before they fill up.
134
00:18:26.220 --> 00:18:40.050
Davide Del Vento: So let me interject here with a comment that bj made on the chat which also was on my mind any it was also somewhat schemed but you know, in the producers light.
135
00:18:40.590 --> 00:18:49.890
Davide Del Vento: And that campaign storage is not a it's not technically purged but it's has a five year retention policy.
136
00:18:50.280 --> 00:19:00.870
Davide Del Vento: Because you know make if you go back to the previous slide when you said, the campaign storage will plateau at 120 petabytes that's because.
137
00:19:01.770 --> 00:19:16.470
Davide Del Vento: We have be buying about as much Sarah you know 20 to 25 petabytes maybe couple of times per year, but each of those purchases has a five year lifetime.
138
00:19:16.920 --> 00:19:38.610
Davide Del Vento: So, starting in 2023 was will start retiring the first batch of disks with the date on it so that data will go away effectively as being a five year period now i'm not sure as Gary said, we have all the tools in place so goofy and everything to make sure.
139
00:19:39.990 --> 00:19:57.420
Davide Del Vento: People on the same bed, but we certainly need to start doing that, after spss is gone, and so we have a little free more free cycles to think about some other decommissioning, that will be the commission's of those discs, so I am stand gary's idea of.
140
00:19:58.620 --> 00:20:01.500
Davide Del Vento: You know, in that section was my also.
141
00:20:02.700 --> 00:20:04.530
Davide Del Vento: Have a purging.
142
00:20:07.170 --> 00:20:07.530
Davide Del Vento: and
143
00:20:09.540 --> 00:20:19.230
Davide Del Vento: But I mean we that we in the trajectory we are on at the moment, there is, there is no option, because.
144
00:20:21.330 --> 00:20:38.310
Davide Del Vento: Because that's the way it is, I mean we could potentially see there, I mean i'm not saying that this is what we are going to do we think it's possible but I don't think we are there to make you know, then wc tree scratch not perched, which means that you're.
145
00:20:39.960 --> 00:20:53.190
Davide Del Vento: I mean everybody, not just your everybody quarter will be much less bigger, so we can provide those generous quarters only because of the purge because, since data is constantly deleted.
146
00:20:54.480 --> 00:21:16.110
Davide Del Vento: The average user is not at their capacity, however, if we don't constantly delete then everybody feels their quota, and so we cannot overlook it and so that means that even with a four time increasing capacity, if we don't do the purge we you know, everybody score I will be.
147
00:21:17.220 --> 00:21:25.320
Davide Del Vento: I mean I don't know the top the number up on my hand, but I would guess you know how for maybe a quarter what you have now on the cheyenne.
148
00:21:29.070 --> 00:21:31.710
Mick Coady: yeah I want to clarify some things to.
149
00:21:33.360 --> 00:21:46.230
Mick Coady: That sizzle is carefully reviewing can seriously considering and reviewing the that stated five year perch policy for campaign storage.
150
00:21:47.640 --> 00:21:51.030
Mick Coady: That it's not universally.
151
00:21:52.890 --> 00:21:56.940
Mick Coady: endorsed by some of us, I let's say that.
152
00:21:58.680 --> 00:22:08.820
Mick Coady: Personally, I feel it's it it'll be very dangerous for sizzle to get into the business of purging data after five years, that on campaign storage.
153
00:22:11.370 --> 00:22:18.330
Mick Coady: Will will will be creating holes in people's data in their projects and.
154
00:22:20.040 --> 00:22:29.520
Mick Coady: So I wouldn't you know we're we still have a little bit, we still have some time to get our arms around this but it's my.
155
00:22:33.000 --> 00:22:35.430
Mick Coady: My personal preference would be.
156
00:22:36.930 --> 00:22:48.480
Mick Coady: That users take care of their own data remove data that they don't need to keep their keep themselves under there they, in their projects underneath their allocations.
157
00:22:49.710 --> 00:23:02.430
Mick Coady: So we'll see we'll see how that plays out but is the other day was saying, eventually, these debt, the the older deaths are going to age often will have to be replaced just for the integrity of the system.
158
00:23:03.870 --> 00:23:06.600
Mick Coady: In replace with newer ones higher capacity.
159
00:23:08.340 --> 00:23:24.300
Mick Coady: We don't have a good we don't have I don't say have a good scheme in place, yet we don't know how we don't have a policy in place and how that's going to how that's going to look for be implemented it's partially what, in large part, why I wanted to start this conversation.
160
00:23:25.710 --> 00:23:29.910
Mick Coady: in earnest now we've talked about it before in the past, but I would.
161
00:23:32.340 --> 00:23:57.510
Mick Coady: we're looking for advice from you all on on issues like that and I don't want to get too far down the campaign storage trail here without considering the without getting back to the question of how, what can we do collectively sizzle and you know you all in the users to make.
162
00:23:58.830 --> 00:24:23.940
Mick Coady: scratch space this large luster space more attractive, or in more better better utilized now is even online yet so it's hard to it's probably not fair to me for me to call that better utilized but we we don't want everything just going into campaign storage and then scratch sitting.
163
00:24:25.470 --> 00:24:27.480
Mick Coady: More idle than.
164
00:24:29.520 --> 00:24:37.230
Mick Coady: or learn more largely unused so Gary can you elaborate a little bit on on what you posted in the chat here.
165
00:24:38.580 --> 00:24:39.960
Gary Strand: Well, what i'm saying there is.
166
00:24:41.220 --> 00:24:49.920
Gary Strand: You know present this information, saying okay here are our options with scratch and campaign storage on casper and direction.
167
00:24:50.670 --> 00:24:56.460
Gary Strand: If you want unstrap unpatched scratch, then each person gets.
168
00:24:57.150 --> 00:25:06.300
Gary Strand: I don't know five terabytes and that's fixed that's all that you get and there will be any more coming So if you need 20 you got to figure out something else.
169
00:25:06.600 --> 00:25:24.420
Gary Strand: Now, if you want, you know if you want to 20 terabytes or 50 or 100 terabytes you know burst type usage, then we got to have the ability to purge scratch, you know if you lay out two or three options, I think you'll probably get a pretty good idea of what users actually want.
170
00:25:25.770 --> 00:25:26.970
Mick Coady: Good and just not leave it.
171
00:25:27.450 --> 00:25:29.700
Gary Strand: Open like I didn't know, we had wide open.
172
00:25:29.820 --> 00:25:38.520
Gary Strand: yeah and the other issue is I don't know how in the world, you can keep people from using unperturbed campaign storage.
173
00:25:39.240 --> 00:25:50.550
Gary Strand: Instead of purged scratch, when all they have to do is put a you know, change your pointer in their code to write to campaign storage, instead of scratch and there's no way you can avoid that well.
174
00:25:50.580 --> 00:25:58.260
Mick Coady: I don't want to i'm sorry if I left the impression that we're looking for to get not for for people not to use campaign source, I mean that.
175
00:25:58.710 --> 00:25:59.790
Gary Strand: Therefore right.
176
00:25:59.880 --> 00:26:00.660
Mick Coady: So I might have.
177
00:26:01.740 --> 00:26:11.730
Mick Coady: My words probably could have been chosen a little bit more carefully so so Mathias and garth have had their hands up for a while let's kill Mathias I think you were first yeah.
178
00:26:12.120 --> 00:26:27.390
Matthias Rempel: I mean, I think one problem is, in our accounting system be charged for computing time, but at least to the user storage, always a ps3 and that always pushes people into the model compute once install everything.
179
00:26:28.320 --> 00:26:37.650
Matthias Rempel: And since there is a substantial cost, with all the storage, we have here I mean it might be worthwhile discussing whether there should be a do accounting system which.
180
00:26:39.090 --> 00:26:51.690
Matthias Rempel: Basically you're being charged, also for the data you store and so people have to make the decision you know if I still unless I can actually come to it more I mean, are there any discussions in that direction if it would make sense.
181
00:26:53.040 --> 00:27:00.180
Mick Coady: I think it makes sense i've been arguing that since I came in car Actually, I find it a little.
182
00:27:02.910 --> 00:27:17.820
Mick Coady: Maybe odd that, given the storage is becoming so so valuable and such a precious commodity, but there's no accounting for it outside of the quota system right so.
183
00:27:19.230 --> 00:27:21.330
Mick Coady: you've got to support with me with this.
184
00:27:24.510 --> 00:27:28.680
Davide Del Vento: Well, we have discussed this option with Dave several times.
185
00:27:30.450 --> 00:27:31.170
Davide Del Vento: But uh.
186
00:27:32.550 --> 00:27:34.860
Davide Del Vento: I don't know why we never committed on.
187
00:27:36.390 --> 00:27:46.920
Davide Del Vento: going in that direction for real and actually you know proof of concept discussions, one reason might be that a it's kind of hard.
188
00:27:48.570 --> 00:28:01.470
Davide Del Vento: To actually calculate that you know doing an actual location that, in practice, with a group that does all these kind of counts, you know it's easy to say well core hours you measure that core hours and you're done.
189
00:28:01.980 --> 00:28:11.820
Davide Del Vento: versus you know terabyte day, then you have to do every day it's check how much fire are many files, you have to match the US and.
190
00:28:12.450 --> 00:28:20.610
Davide Del Vento: it's more complicated to to measure that might be the reason but uh but I agree that will be more desirable.
191
00:28:20.970 --> 00:28:34.110
Davide Del Vento: mean, in my opinion, though in the situation that we are headed now, it will be even better if we just did our computers totally free and only storage speed, but i'm probably going too far, now.
192
00:28:41.460 --> 00:28:47.310
Davide Del Vento: So i'm sure these conversation will happen again with with the management well the then you'll see some will be here, but.
193
00:28:49.230 --> 00:28:51.750
Davide Del Vento: we'll let you know what happens, but I don't think.
194
00:28:52.950 --> 00:28:55.650
Davide Del Vento: It is something that will happen in the ratio time.
195
00:29:00.120 --> 00:29:03.750
Mick Coady: or alright girth sorry any weights on.
196
00:29:04.890 --> 00:29:06.570
Garth D'Attilo: that's that's fine.
197
00:29:07.890 --> 00:29:18.900
Garth D'Attilo: You know I couldn't stay quiet, the whole time the so you know my understanding was that the campaign storage purge was off the table entirely because you can't Swiss cheese stuff so i'd like some confirmation on that.
198
00:29:19.950 --> 00:29:26.040
Garth D'Attilo: Before we probably go any further, because that is that's a non starter, I think, for everybody doing work with campaign storage.
199
00:29:26.970 --> 00:29:29.610
Mick Coady: And that is what what's an insert.
200
00:29:30.690 --> 00:29:35.760
Garth D'Attilo: A purge policy on campaign, I mean you're gonna if you Swiss cheese data sets.
201
00:29:36.360 --> 00:29:44.700
Garth D'Attilo: Community data sets other day, anyway, I thought it sounded from this meeting, like that was back on the table, and I was pretty sure that that have been taken off at some point.
202
00:29:45.060 --> 00:29:54.630
Garth D'Attilo: So I just love some clarification, if you guys have to go back and meet to make sure that's either it's not going to happen, or we bring it back up and discuss the reasons why it's probably a really horrible idea.
203
00:29:55.770 --> 00:29:56.760
Garth D'Attilo: For that to happen.
204
00:29:57.930 --> 00:30:00.150
Mick Coady: I promise i'll take that back in.
205
00:30:01.350 --> 00:30:09.450
Mick Coady: express how to horrible idea, most of us think that is okay I can't promise what policies, going to be put in place.
206
00:30:09.960 --> 00:30:24.300
Mick Coady: you've removed the reason that it's still even out there in the ether, is that when you remember when campaign stories first came online That was good, that was the stated purpose of it right that it was supposed to be for.
207
00:30:26.250 --> 00:30:29.460
Mick Coady: data that was on a.
208
00:30:30.600 --> 00:30:41.310
Mick Coady: Project timescales publication timescales, which was estimated to be about five years, well, I think many of us recognize that campaign storage is.
209
00:30:42.570 --> 00:30:42.930
Mick Coady: The.
210
00:30:44.160 --> 00:30:46.920
Mick Coady: it's not used like that its usage.
211
00:30:48.750 --> 00:30:54.330
Mick Coady: has expanded beyond that original intent so.
212
00:30:55.980 --> 00:31:15.150
Mick Coady: I know Pam before she left was strongly arguing against the policy and she convinced me like right away yeah I mean it like you said, the Swiss you can't Swiss cheese, a project like that now, however, the.
213
00:31:16.380 --> 00:31:23.760
Mick Coady: usage can't continue to grow at the pace, it has without some discipline or.
214
00:31:25.740 --> 00:31:35.790
Mick Coady: User intervention, maybe I guess, and this is my opinion right that eventually files are going to date is going to have to be removed by users.
215
00:31:39.390 --> 00:31:42.930
Mick Coady: Because will will they are, we will just run out of space.
216
00:31:44.130 --> 00:31:44.520
Mick Coady: Right.
217
00:31:45.960 --> 00:31:53.970
Garth D'Attilo: yeah and I think that it's you know if we're going to use it as HP SS replacement or long term data storage for data sets that have historical significance let's say.
218
00:31:54.390 --> 00:32:02.640
Garth D'Attilo: That or they go beyond five years for people looking back in time to analyze you know previous model output, or what have you, which is a pretty common use case, at least for a calm.
219
00:32:03.240 --> 00:32:08.730
Garth D'Attilo: Then you know you need to probably default to the same sort of situation maven project spaces.
220
00:32:09.480 --> 00:32:17.580
Garth D'Attilo: You tell everybody what they're using the big users have to do something about it, and you know to gary's point then goofy has to it would be great if that's like hey this date is this old.
221
00:32:17.970 --> 00:32:24.510
Garth D'Attilo: You have so much of it, you haven't touched in this long and you know, obviously there's this sort of administrative overhead attached to that.
222
00:32:24.750 --> 00:32:27.510
Garth D'Attilo: But I think that's probably much better than the alternative which is.
223
00:32:27.750 --> 00:32:32.370
Garth D'Attilo: Oh whoops that's just gone, but not all of it only certain files and we can't really tell you which ones so.
224
00:32:32.850 --> 00:32:48.390
Garth D'Attilo: yeah so that I guess that's that's the number one point there and I know well, we can further discuss that but I yeah I would love some clarity to make sure that that we're not going to do that, or we need to make different plans for our long term holdings on data.
225
00:32:49.410 --> 00:32:52.290
Garth D'Attilo: for things that can't be removed Community data sets reference sets.
226
00:32:53.880 --> 00:32:58.140
Garth D'Attilo: That haven't been tagged properly, so that they survive forever and you know there's a big risk to that.
227
00:32:59.070 --> 00:33:01.290
Garth D'Attilo: hmm um the other, you know.
228
00:33:02.880 --> 00:33:20.460
Garth D'Attilo: Obviously we're going to run out of space on campaign storage most likely when you open it up one of the things we discussed before was the scratch base and making it very easy for those big users or easier to request temporary increases to the amount of scratch base that they have.
229
00:33:21.930 --> 00:33:25.770
Garth D'Attilo: so that they could say get their hundred terabytes and run their.
230
00:33:26.640 --> 00:33:35.670
Garth D'Attilo: model, and do you know their analysis within six months, I think that's totally reasonable and there's a training aspect to that, of course, to make sure users kind of fall, along with that, but.
231
00:33:36.180 --> 00:33:45.090
Garth D'Attilo: I think if people have that ability to increase their current allotment, then you probably be able to get many more to use it, it is faster right is that part of the.
232
00:33:45.930 --> 00:33:47.490
Garth D'Attilo: As Lester file system faster.
233
00:33:48.240 --> 00:33:55.290
Mick Coady: um I would have to defer to the folks who knew much more about that than I do.
234
00:33:56.730 --> 00:33:57.690
Davide Del Vento: This in here.
235
00:33:58.770 --> 00:34:05.850
Davide Del Vento: I think signals push remain standing is that it is faster but but am the expert on that is it so.
236
00:34:07.170 --> 00:34:10.170
Sidd: it's not in any significant way God.
237
00:34:10.410 --> 00:34:21.420
Sidd: OK, the part of the reason we are trying to encourage last batch is because of its size it's a huge one.
238
00:34:22.500 --> 00:34:34.200
Sidd: We have paid for it, the reason we may choose to be that big is We wanted the performance, so their performance and capacity go hand in hand.
239
00:34:36.090 --> 00:34:36.810
Sidd: So.
240
00:34:38.310 --> 00:34:38.670
Sidd: yeah.
241
00:34:40.320 --> 00:34:44.970
Sidd: that's pretty much the context or thinking from the sizzle side that we here.
242
00:34:46.050 --> 00:34:47.970
Sidd: Once we get.
243
00:34:48.990 --> 00:34:50.400
Sidd: rid of these resources.
244
00:34:51.810 --> 00:34:52.380
Sidd: uses.
245
00:34:54.030 --> 00:34:55.470
Sidd: less likely that scratching.
246
00:34:58.620 --> 00:34:59.040
Garth D'Attilo: yeah.
247
00:35:01.140 --> 00:35:01.710
Garth D'Attilo: Go ahead, David.
248
00:35:02.520 --> 00:35:18.240
Davide Del Vento: I was asking for clarification from seed about the performance when you see it's more performance you mean as an aggregated performance but single stream performance, which is what they use or my see is to see more or less.
249
00:35:18.930 --> 00:35:30.270
Sidd: that's precisely the point so practically every it will be practically the same so that's why we can't make performance argument.
250
00:35:30.900 --> 00:35:45.990
Sidd: To use scratch, it is just the capacity part you may have force users to use less can being more of scratch, by restricting their code onto campus campaigns do it, but then.
251
00:35:48.150 --> 00:35:50.670
Sidd: Maybe that is a limit so on so far.
252
00:35:53.970 --> 00:36:05.130
Garth D'Attilo: Okay, well, and in that case you know, there is you, then I guess the the sales pitches that you won't be continually hassled by data managers, like myself, if you're using too much space.
253
00:36:06.360 --> 00:36:09.930
Garth D'Attilo: Because you will you'll have an auto purge on.
254
00:36:11.100 --> 00:36:19.170
Garth D'Attilo: On the scratch base but yeah you know you're probably going to run out of all the space really I mean that's just the nature of what we do here.
255
00:36:20.040 --> 00:36:28.140
Garth D'Attilo: So it's just a matter of time, so the ultimate point is really well Okay, what do you do when that happens what's the contingency plan for when that does happen.
256
00:36:29.160 --> 00:36:34.860
Garth D'Attilo: And of course you bring up a great point when you replace this where does that data go to where's the staging area, do you have one if there isn't.
257
00:36:35.550 --> 00:36:42.600
Garth D'Attilo: yeah accent, you need to hold back some capacity to make sure that's a possibility, but I think we need to start planning for an eventuality that everything fills up.
258
00:36:42.990 --> 00:36:46.890
Garth D'Attilo: And once campaign storage is pretty fold and hey you know what scratch base is going to be used.
259
00:36:47.730 --> 00:37:02.850
Garth D'Attilo: Even more, but you know people are going to just change the directory name in there output and then that's that's it and it's going to do the hockey stick thing I mean you know i'm I don't know the future, but the future looks like a probable hockey stick yeah.
260
00:37:03.930 --> 00:37:10.500
Mick Coady: So in this is really good I guess two questions or a question you want to make first of all.
261
00:37:12.570 --> 00:37:20.400
Mick Coady: let's remember that we're just having a conversation here right and we're trying to get some guidance and information from from you all.
262
00:37:21.060 --> 00:37:40.260
Mick Coady: To help guide our policies and help shape our policies, so the last thing I think any of us need is for the word word to come out of this meeting saying sizzles going to start purging campaign storage, that would be.
263
00:37:42.630 --> 00:37:45.150
Mick Coady: As far as i'm concerned, nothing can be further from the truth.
264
00:37:46.260 --> 00:37:53.340
Mick Coady: until further unless we come back with some specific information, otherwise, so please don't.
265
00:37:55.590 --> 00:38:08.130
Mick Coady: leak that or start having that kind of a conversation with your users that that's what I would ask of you, and then the say I need a clarification from you garth about making it easier to get.
266
00:38:09.180 --> 00:38:18.570
Mick Coady: extensions to scratch space quotas is it being Is it the users think it's difficult now.
267
00:38:19.920 --> 00:38:21.390
Mick Coady: Is it an onerous process.
268
00:38:21.540 --> 00:38:31.230
Garth D'Attilo: I don't know eleanor's i've never actually requested I don't know if there's a great awareness, but you're looking at ways that you might want if it's awareness issue but you're looking at ways that you can somehow reduce the.
269
00:38:31.530 --> 00:38:39.330
Garth D'Attilo: Pressure on campaign storage and one of those things would be well i'm going to have 100 terabytes output I don't need to save it forever, but if my quota is only 15.
270
00:38:39.990 --> 00:38:47.190
Garth D'Attilo: or whatever it turns out to be then I have no other place to put this but on campaign storage right says project space goes away so.
271
00:38:47.640 --> 00:39:01.170
Trude Eidhammer: Are you are utilize the extra scratch place a lot of time creating a lot of data, a lot of moving around, and it has not been difficult to get extra scratch place space and to also extend it.
272
00:39:02.250 --> 00:39:05.460
Trude Eidhammer: that's not so far, it has not been a difficult process.
273
00:39:07.350 --> 00:39:23.790
Davide Del Vento: I think that a historically in I think is still is it like that, in our documentation for that historical reason we have not advertised that, because in the past we didn't have so much opportunity to increase it, as we have the moment.
274
00:39:26.010 --> 00:39:41.490
Davide Del Vento: So you know it's a word word of mouth or people that say, well, let me ask and see what our bands, but I don't think it's on our documentation that you know you can request it so maybe that's something we need to adjust and change.
275
00:39:42.630 --> 00:39:56.910
Garth D'Attilo: And that that would be my recommendation is, if you want some work and civil do to encourage Rachel scratch base will advertise that you can get more than the basic allotment and I think you'll get a lot more users to not just dump into campaign storage and then forget about it forever.
276
00:39:59.190 --> 00:39:59.580
Okay.
277
00:40:03.210 --> 00:40:03.540
Mick Coady: What.
278
00:40:05.760 --> 00:40:11.010
Mick Coady: Is it, you know bj was with us, I think he's gone now but.
279
00:40:12.360 --> 00:40:14.400
Mick Coady: I see bj bj do you knew if.
280
00:40:15.600 --> 00:40:28.110
Mick Coady: I don't recall if we have anything in our documentation today or have had about you, if you need more space simply contact to contact us.
281
00:40:30.600 --> 00:40:44.310
bjsmith: I couldn't put my finger on, but I think we probably do mention that somewhere, probably in the in the glade documentation and i'm pretty sure there's a there's a link in there in the service desk.
282
00:40:45.690 --> 00:40:48.090
bjsmith: For a ticket maybe specifically requesting.
283
00:40:49.650 --> 00:40:50.580
bjsmith: An increase.
284
00:40:51.840 --> 00:40:58.770
bjsmith: On homie to have, I think it's it's there, I mean we haven't got out of our way to emphasize that you know, encourage people to you know.
285
00:40:59.880 --> 00:41:07.560
bjsmith: requested, you know just randomly or anything but it's that's their people know they can do it yeah.
286
00:41:08.460 --> 00:41:18.030
Garth D'Attilo: And maybe I was thinking more like an introduction to the ratio right where you tell people what the intended purposes for each of the storage spaces that are there, or something along those lines, so that you reiterate the.
287
00:41:18.330 --> 00:41:26.400
Garth D'Attilo: hey this is how we'd like to see you use it, so we don't run out, and this, this is what happens if we do, and you know here's how to be a good, you know, this is a good opportunity, perhaps to.
288
00:41:27.510 --> 00:41:36.690
Garth D'Attilo: To to give out that information, a learning opportunity for for our users, that you know, have to be more data conscious going forward, you know there's no other option here.
289
00:41:37.680 --> 00:41:42.570
bjsmith: As that sounds like a fine idea to me not sure who started out but.
290
00:41:43.890 --> 00:41:49.320
bjsmith: You, and you know when when campaign started initially came out that was the thing about the five year.
291
00:41:50.850 --> 00:41:56.400
bjsmith: retention period was this was discussed and included in some of those training things.
292
00:41:57.600 --> 00:41:58.440
bjsmith: Obviously.
293
00:41:59.550 --> 00:42:05.640
bjsmith: Not a pipe still not a popular idea, but this one reason I asked about it earlier is because.
294
00:42:06.720 --> 00:42:08.700
bjsmith: Someone mentioned, I think Gary mentioned the.
295
00:42:10.050 --> 00:42:19.980
bjsmith: scribe campaign started just purged and they're conflicted precisely it was something I knew we had in our documentation, so I want to get that clarified also.
296
00:42:22.950 --> 00:42:23.670
bjsmith: stopped out.
297
00:42:24.240 --> 00:42:28.170
Mick Coady: that's it thanks bj know if if that's still.
298
00:42:29.370 --> 00:42:39.810
Mick Coady: We want our documentation to be as accurate as possible, obviously right because it could, if it's if it's incorrect, it could lead users to.
299
00:42:41.610 --> 00:42:49.500
Mick Coady: Bad bad bad practices, so what will check will check into that and.
300
00:42:50.970 --> 00:42:59.970
Mick Coady: I will promise that by the honor before our next meeting next month that will do our best.
301
00:43:01.530 --> 00:43:12.900
Mick Coady: said Java day and I will do our best to get a firm answer on this old five year the business on campaign storage data okay so like.
302
00:43:13.410 --> 00:43:30.810
Mick Coady: Again i'm just asked let's just keep this amongst ourselves this conversation now i'm pretty confident that that's not going to happen that we're not going to be purging data just because it hit a five year from create date the.
303
00:43:32.010 --> 00:43:32.460
Mick Coady: limit.
304
00:43:34.950 --> 00:43:36.840
Mick Coady: But i've been wrong before but.
305
00:43:37.950 --> 00:43:38.940
Mick Coady: I pretty much stake.
306
00:43:40.530 --> 00:43:44.310
Mick Coady: My reputation on that what will reputation yeah.
307
00:43:45.570 --> 00:43:50.160
Garth D'Attilo: That was coming one last thing and that's a doug's comment in the.
308
00:43:50.520 --> 00:44:00.810
Garth D'Attilo: In the chat you know and we it's a huge resource, be able to take advantage of the RTA and whatever when you're trying to put in, or you have a Community data set and you need to.
309
00:44:01.320 --> 00:44:10.680
Garth D'Attilo: push that off somewhere that won't be purged and has you know dap access and all those sorts of things so that's a huge resource that probably also might need to be spread around or you know what what.
310
00:44:11.430 --> 00:44:22.410
Garth D'Attilo: What warrants you know what is a Community reference data set how many people have to use it does your data qualify for these things, these are all resources that we have available that could help reduce the.
311
00:44:22.950 --> 00:44:30.690
Garth D'Attilo: The push on campaign storage, as long as people know that they are options, and you know it's a it would be another.
312
00:44:31.080 --> 00:44:38.160
Garth D'Attilo: educational opportunity I think just tell people hey we have the RTA or these other options which are also, and this is what they're intended for.
313
00:44:38.520 --> 00:44:47.940
Garth D'Attilo: You know, with all the storage options changing it'd be very helpful to have pointers to Okay, this is, this is what we'd like you to use it for this is what qualifies for inclusion.
314
00:44:48.510 --> 00:44:57.330
Garth D'Attilo: Not an easy process, I know, but you know I think there's going to be a lot of confusion with well where should we put stuff campaign storage is just a you know, an empty storage space Well, yes and no.
315
00:44:58.170 --> 00:45:06.600
Garth D'Attilo: Again right so there's an opportunity here, I think, just to clarify what sizzle would like to see, and you know, make sure everybody can be a good data Stewart.
316
00:45:07.710 --> 00:45:12.690
Mick Coady: yeah and thanks for your your posts there I think that's.
317
00:45:13.740 --> 00:45:15.900
Mick Coady: was very timely appreciate.
318
00:45:19.980 --> 00:45:21.270
Mick Coady: So, in terms of.
319
00:45:21.840 --> 00:45:23.160
Mick Coady: Again I don't want to keep.
320
00:45:24.180 --> 00:45:35.700
Mick Coady: harping on the five year thing but remember that camping storage first came online if i'm if I remember correctly, in the summer of 2018 so.
321
00:45:36.840 --> 00:45:54.210
Mick Coady: we're we're still pretty much two years away from the oldest data sets on there even coming at risk, so there's there's certainly no cause for panic or alarm at this point, I just wanted to throw that out there.
322
00:45:55.620 --> 00:45:56.100
Mick Coady: Okay.
323
00:45:58.530 --> 00:45:59.160
Mick Coady: let's see.
324
00:46:00.660 --> 00:46:04.110
Mick Coady: yeah let's get past HP SS first no kidding.
325
00:46:05.670 --> 00:46:13.980
Mick Coady: deke ville I think is going to buy a nice bottle of champagne when that's done on October 1.
326
00:46:15.210 --> 00:46:30.570
Mick Coady: Anything else that we're kind of coming up on an hour, and I think it's been pretty good conversation you does anybody else have anything they want to contribute here any comments you want to make now before well we're all still here.
327
00:46:35.940 --> 00:46:49.770
Garth D'Attilo: It make what one last thing the last question or not the last question, but they using P amp D project based scratch basis, I know, in the past, and these meetings, there have been those people that wanted better easier access to sharing sometimes the setting up the permissions.
328
00:46:50.220 --> 00:46:53.520
Garth D'Attilo: For campaign storage is perhaps a little more difficult than it would be.
329
00:46:54.060 --> 00:46:55.530
Garth D'Attilo: On just a standard.
330
00:46:56.670 --> 00:47:09.780
Garth D'Attilo: storage space like scratch, and so you know, I think, yes project based scratch basis could be a win for those projects that do need a lot of collaboration and maybe more space temporarily to get something done without having to manage it through.
331
00:47:10.890 --> 00:47:16.830
Garth D'Attilo: The groups and people db or what have you, so I think that's a really good idea to to explore.
332
00:47:17.730 --> 00:47:30.600
Garth D'Attilo: And maybe just on a case by case basis, because I imagine this, this would be a time limited thing from time to time for a particular project, but that that sounds like a good way to try and reduce the pressure on command campaign storage as well.
333
00:47:31.140 --> 00:47:40.260
Mick Coady: Okay yeah I this is kind of a new idea that we just has come up with since since we all got together last month.
334
00:47:41.220 --> 00:47:42.900
Mick Coady: In I think he.
335
00:47:43.920 --> 00:47:45.330
Mick Coady: Is in the past we.
336
00:47:46.350 --> 00:47:48.690
Mick Coady: Because of limitations with GPS and.
337
00:47:50.820 --> 00:47:58.860
Mick Coady: In in rather more limited on the available space right 15 petabytes for several thousand users.
338
00:47:59.880 --> 00:48:02.280
Mick Coady: We I think there was a concern about.
339
00:48:03.540 --> 00:48:04.350
Mick Coady: Trying trying to.
340
00:48:06.390 --> 00:48:26.580
Mick Coady: serve two masters, if you will, well with this large I know it probably won't seem large five years from now, when it's starting to age off, but with a 60 petabyte luster file system, I think we may have opportunities to carve out.
341
00:48:27.870 --> 00:48:28.530
Mick Coady: Parallel.
342
00:48:29.880 --> 00:48:44.460
Mick Coady: systems or universes here right one, one for users and one for project, maybe they won't be equal size, but I think there's a lot of possibilities that could be could be explored and put on the table here so.
343
00:48:45.480 --> 00:48:49.560
Mick Coady: It to me it sounds like a good idea, like, for the reasons you.
344
00:48:50.730 --> 00:48:51.750
Mick Coady: just listed garth.
345
00:49:00.720 --> 00:49:04.920
Mick Coady: give everybody another minute or so to stew or think about.
346
00:49:06.180 --> 00:49:10.890
Mick Coady: contemplate your ponder and if.
347
00:49:11.970 --> 00:49:14.880
Mick Coady: If there isn't anything else we'll wrap things up.
348
00:49:22.530 --> 00:49:23.400
Mick Coady: Okay, thanks Gary.
349
00:49:27.540 --> 00:49:29.220
Mick Coady: it's been pretty quiet today.
350
00:49:30.900 --> 00:49:31.500
Mick Coady: that's okay.
351
00:49:35.100 --> 00:49:36.630
Mick Coady: All right, all of.
352
00:49:37.800 --> 00:49:40.140
Mick Coady: i'll thank everybody again for showing up today.
353
00:49:41.430 --> 00:49:42.270
Mick Coady: we've got some.
354
00:49:44.070 --> 00:49:50.520
Mick Coady: said bj duvernay and I have some action items to.
355
00:49:51.960 --> 00:49:59.970
Mick Coady: Work on before we get back together again if it seems appropriate i'll just send communicate those with you, through email.
356
00:50:01.080 --> 00:50:02.400
Mick Coady: And look for.
357
00:50:03.450 --> 00:50:10.650
Mick Coady: These slides and the recording of this meeting on on the wiki i'll let you know when that's out there as well, so.
358
00:50:11.790 --> 00:50:16.560
Mick Coady: Thanks everybody have a good rest of your day and take care.
359
00:50:20.340 --> 00:50:20.670
Matthias Rempel: bye.