Author |
Message |
Michael Goetz Volunteer moderator Project administrator
 Send message
Joined: 21 Jan 10 Posts: 13513 ID: 53948 Credit: 236,922,854 RAC: 0
                           
|
If you look near the bottom of the menu on the left side of this page, you'll see a new link, "Task status".
There's charts on that page showing the number of outstanding tasks in each subproject, and the recent execution times for each subproject.
Most of the charts use a logarithmic scale on the y-axis due to the large differences between different subprojects.
Please note that execution times are expressed in hours. Times less than one hour are shown in thousandths of an hour, so "500m" is half an hour, or 30 minutes.
You may thank JimB for the idea, and blame me for the execution.
____________
My lucky number is 75898524288+1 |
|
|
|
Great addition! |
|
|
|
Looks good :) Will be more interesting as data builds up for longer term trends.
I have to ask, what does "task in progress" mean? Does it mean "has been sent out but not yet returned"? As opposed to, new tasks sent out, or tasks received?
For example, say I do short units (SGS, PPSE) without cache, and return them as soon as they're finished. Would that show up as one "outstanding" unit per core, even though I might do hundreds per day?
Reason for asking is I think it is a more interesting number to see how many units are "done" than how many are floating out there which can vary due to cache and unit lengths. |
|
|
Michael Goetz Volunteer moderator Project administrator
 Send message
Joined: 21 Jan 10 Posts: 13513 ID: 53948 Credit: 236,922,854 RAC: 0
                           
|
I have to ask, what does "task in progress" mean? Does it mean "has been sent out but not yet returned"? As opposed to, new tasks sent out, or tasks received?
Tasks that have been sent out to a host but not yet returned.
For example, say I do short units (SGS, PPSE) without cache, and return them as soon as they're finished. Would that show up as one "outstanding" unit per core, even though I might do hundreds per day?
Correct. It would shows as one task. This is the number of outstanding tasks, not the number of tasks done each day.
Reason for asking is I think it is a more interesting number to see how many units are "done" than how many are floating out there which can vary due to cache and unit lengths.
These statistics are gathered at 5 minute intervals, and the database query needed to gather the data for "how many do we do each day" is (currently) quite expensive. I may add this if I'm able to optimize the SQL to an acceptable level.
____________
My lucky number is 75898524288+1 |
|
|
|
Looks good and thanks for getting it setup.
[What is/will be the update timeframe?]
EDIT: sorry didn't see your response before I posted.
Thanks Rick
____________
@AggieThePew
|
|
|
|
These statistics are gathered at 5 minute intervals, and the database query needed to gather the data for "how many do we do each day" is (currently) quite expensive. I may add this if I'm able to optimize the SQL to an acceptable level.
Thanks for the clarifications. For how many tasks we do per day, would it be sufficient to only update it once per day? Or is it still expensive to do that?
To further complicate matters, I was thinking how would you count completed tasks anyway? Returned tasks may include error, aborts and timeouts. Other tasks may look ok but later invalidate. Waiting to validate would delay the stats. I was thinking it may be adequate just to count correctly formatted results, like the stats for challenges, even if they may later be invalidated. I'm assuming the invalidated units would be relatively low in number so not significant as far as these stats are concerned. |
|
|
Michael Goetz Volunteer moderator Project administrator
 Send message
Joined: 21 Jan 10 Posts: 13513 ID: 53948 Credit: 236,922,854 RAC: 0
                           
|
These statistics are gathered at 5 minute intervals, and the database query needed to gather the data for "how many do we do each day" is (currently) quite expensive. I may add this if I'm able to optimize the SQL to an acceptable level.
And this is done. Wait a bit for a few data points to be collected and you'll start seeing lines appearing on the new charts.
____________
My lucky number is 75898524288+1 |
|
|
Michael Goetz Volunteer moderator Project administrator
 Send message
Joined: 21 Jan 10 Posts: 13513 ID: 53948 Credit: 236,922,854 RAC: 0
                           
|
To further complicate matters, I was thinking how would you count completed tasks anyway? Returned tasks may include error, aborts and timeouts. Other tasks may look ok but later invalidate. Waiting to validate would delay the stats. I was thinking it may be adequate just to count correctly formatted results, like the stats for challenges, even if they may later be invalidated. I'm assuming the invalidated units would be relatively low in number so not significant as far as these stats are concerned.
"Completed tasks" means tasks returned successfully (i.e., it didn't report a computation error or another problem) and is either pending validation, validated, or inconclusive. If it failed validation it's not counted. It's similar to the rules for challenge points.
____________
My lucky number is 75898524288+1 |
|
|
Michael Goetz Volunteer moderator Project administrator
 Send message
Joined: 21 Jan 10 Posts: 13513 ID: 53948 Credit: 236,922,854 RAC: 0
                           
|
I have added a "New Users" chart that shows the number of new users within the last 24 hours. It shows two metrics: all users, and "active users". Active users are users that have started to run at least one task.
I suspect that non-active users are mostly spambots, but I have no data to support that assumption.
____________
My lucky number is 75898524288+1 |
|
|
Michael Goetz Volunteer moderator Project administrator
 Send message
Joined: 21 Jan 10 Posts: 13513 ID: 53948 Credit: 236,922,854 RAC: 0
                           
|
I recently added a T-Flops/s chart to the new page. It shows the current processing power in use by all of you in support of PrimeGrid. It's currently about 1.2 Peta-Flops per second.
I wouldn't try comparing our Flops estimate to anyone else's. Chances are any other BOINC site calculates them differently than we do, and who knows how non-BOINC sites calculate this number. The purpose of the chart is mostly to see how our processing power changes over time, especially during challenges as well as long term seasonal and yearly trends.
I can, however, tell you how I'm doing the calculation:
200 credits is, by definition, equivalent to a day's worth of processing by a computer doing 1 Giga-Flop per second. Our Flops estimate is therefore based on the number of credits granted. I think we have one of the most accurate credit systems, so it stands to reason that the Flops estimate should be pretty accurate also.
Note that this number includes only BOINC tasks. It does not include the processing power involved in either manual sieving or PRPNet.
____________
My lucky number is 75898524288+1 |
|
|
|
If you look near the bottom of the menu on the left side of this page, you'll see a new link, "Task status".
There's charts on that page showing the number of outstanding tasks in each subproject, and the recent execution times for each subproject.
This information is absolutely excellent and allows one to see how their performance for a particular subproject is relative to the group as well. |
|
|
|
200 credits is, by definition, equivalent to a day's worth of processing by a computer doing 1 Giga-Flop per second.
What makes me wonder -- is The Long Job Credit Bonus considered in or out of these 200 cobblestones?
____________
I'm counting for science,
Points just make me sick. |
|
|
Michael Goetz Volunteer moderator Project administrator
 Send message
Joined: 21 Jan 10 Posts: 13513 ID: 53948 Credit: 236,922,854 RAC: 0
                           
|
200 credits is, by definition, equivalent to a day's worth of processing by a computer doing 1 Giga-Flop per second.
What makes me wonder -- is The Long Job Credit Bonus considered in or out of these 200 cobblestones?
Out.
I'm intentionally gathering the GFLOPS statistics slightly differently than the standard (and easier) BOINC method in order to avoid inflating the metric because of the bonuses.
____________
My lucky number is 75898524288+1 |
|
|
|
I think this is a great addition. I've always been curious as to which projects get more use than others. I would like to also see the statistics on number of tasks completed per day, but do understand how that would be more intensive to query from the database.
____________
|
|
|
Michael Goetz Volunteer moderator Project administrator
 Send message
Joined: 21 Jan 10 Posts: 13513 ID: 53948 Credit: 236,922,854 RAC: 0
                           
|
I would like to also see the statistics on number of tasks completed per day, but do understand how that would be more intensive to query from the database.
Those charts are already there. :)
____________
My lucky number is 75898524288+1 |
|
|
|
I would like to also see the statistics on number of tasks completed per day, but do understand how that would be more intensive to query from the database.
Those charts are already there. :)
I missed your post saying it had been done, and didn't scroll far enough down the page to find it. Very cool! I'm moderately surprised that there are more ESP sieve tasks finished per day than PPS sieve tasks. It's very interesting to see how these trends evolve over time.
____________
|
|
|
RafaelVolunteer tester
 Send message
Joined: 22 Oct 14 Posts: 885 ID: 370496 Credit: 334,085,845 RAC: 0
                  
|
I would like to also see the statistics on number of tasks completed per day, but do understand how that would be more intensive to query from the database.
Those charts are already there. :)
I missed your post saying it had been done, and didn't scroll far enough down the page to find it. Very cool! I'm moderately surprised that there are more ESP sieve tasks finished per day than PPS sieve tasks. It's very interesting to see how these trends evolve over time.
I actually find it pretty normal. PPS runs one pre GPU, so usually 1 or 0 (I can imagine a plethora of older PCs falling into this category). ESP, on the other hand, runs 1 per core, so usually 2 or more (and it's the most advised project for that plethora of old PCs, as well as AMD's). Besides, now that GFN has some pretty quick tasks, I'd expect GPUs to focus on those instead. |
|
|
Michael Goetz Volunteer moderator Project administrator
 Send message
Joined: 21 Jan 10 Posts: 13513 ID: 53948 Credit: 236,922,854 RAC: 0
                           
|
I would like to also see the statistics on number of tasks completed per day, but do understand how that would be more intensive to query from the database.
Those charts are already there. :)
I missed your post saying it had been done, and didn't scroll far enough down the page to find it. Very cool! I'm moderately surprised that there are more ESP sieve tasks finished per day than PPS sieve tasks. It's very interesting to see how these trends evolve over time.
ESP Sieve tasks are MUCH smaller tasks than PPS sieve tasks. On a typical CPU, an ESP sieve task takes tens of minutes, but a PPS sieve task takes tens of hours. Also, ESP-Sieve is the default CPU task for new users who don't explicitly select anything. Put the two together and the result is a lot of CPU cores running a lot of relatively short ESP sieve tasks.
Which, of course, is exactly what we want. ESP was previously only sieved to n=10M, and with ESP having moved over to BOINC the LLR testing is now at almost n=9M. We need a new sieve file created for n>10M. And, no, that's not a call-to-arms for everyone to start sieving ESP. The ESP sieve is coming along nicely.
____________
My lucky number is 75898524288+1 |
|
|
|
I would like to also see the statistics on number of tasks completed per day, but do understand how that would be more intensive to query from the database.
Those charts are already there. :)
I missed your post saying it had been done, and didn't scroll far enough down the page to find it. Very cool! I'm moderately surprised that there are more ESP sieve tasks finished per day than PPS sieve tasks. It's very interesting to see how these trends evolve over time.
I actually find it pretty normal. PPS runs one pre GPU, so usually 1 or 0 (I can imagine a plethora of older PCs falling into this category). ESP, on the other hand, runs 1 per core, so usually 2 or more (and it's the most advised project for that plethora of old PCs, as well as AMD's). Besides, now that GFN has some pretty quick tasks, I'd expect GPUs to focus on those instead.
I just always thought that we had a large influx of GPU users come over from other projects because we were one of the only projects with GPU support, and PPS Sieve had some of the best credit rates, but I guess that effect has diminished in recent years or wasn't as large as I had always thought.
____________
|
|
|
|
Quick question, on the completed tasks charts is that tasks that have been turned in and may or may not be pending validation or is it just tasks that have been turned in and validated?
Cheers Rick |
|
|
Michael Goetz Volunteer moderator Project administrator
 Send message
Joined: 21 Jan 10 Posts: 13513 ID: 53948 Credit: 236,922,854 RAC: 0
                           
|
Quick question, on the completed tasks charts is that tasks that have been turned in and may or may not be pending validation or is it just tasks that have been turned in and validated?
Cheers Rick
It counts tasks that have been received successfully, but not necessarily validated yet. "Validated", "Pending", and "Inconclusive" count towards this statistic.
____________
My lucky number is 75898524288+1 |
|
|
|
Quick question, on the completed tasks charts is that tasks that have been turned in and may or may not be pending validation or is it just tasks that have been turned in and validated?
Cheers Rick
It counts tasks that have been received successfully, but not necessarily validated yet. "Validated", "Pending", and "Inconclusive" count towards this statistic.
Thanks for the update! |
|
|