Figures shown for the last few days seem to be significantly lower than on previous days, and they don't really correspond with what's shown on the graph. I think the graph figures are correct. Can you check please?
Thanks for reporting this, Graham. It looks like a stats update process might have gotten stuck following server maintenance. It will be kicking off again soon and we'll know in a day or so whether the problem is fixed. If not, we'll head into debugging round 2.
The fix is in. You should now see a giant spike in your chart as more than a week's worth of points have flooded into the latest stats update. Thanks again for reporting.
It was a good opportunity to make a number of updates to the stats processing system that had been on deck!
Hi all. We moved some key backend infrastructure to a new datacenter, which impacted different services to various extents from Sunday to Tuesday. So there's now a bit of a processing backlog, but this should sort itself in the next day or two. No credits or data should be lost. So if you're still seeing missing stats Friday eve or over the weekend, please post back.
My daily average did a one time jump sometime over last weekend. And then dropped in half, and is currently reading zero.
The chart of daily production never increased when the daily average did over the weekend. It appears unchanged and near but above zero.
My computers have been on full time, a total of 16 processors. The only downtime for them occurred over this weekend when Rosetta and World Community Grid both ran dry as far as work.
It was the production before this last weekend that I still wonder about.
So, you said to mention it if your reporting didn't start to update again, last week I think, when the fix was implemented. My stats have not updated since then, all week, here anyway.
From the BOINC, my total production for Charity Engine is 136,655 and my average production is 974.25,
From your webpage, total production fro Charity Engine is 123,156 with 0 daily average, since last week.
So I will just watch that total from the BOINC client instead of the numbers from here.
Most recently, I have had the majority of units from World Community Grid. Since they have issues with work running out, I have added in Rosetta again. They also have issues at times. I was hoping between them I would be covered. From my graph of daily work, I had been doing fine, until a bit before I read here that others were having production issues with daily totals dropping to zero. At that time, I had been fine. When I saw that you had said the issue had been fixed, my daily stats started to fall on your page.
When I started Charity Engine, I connected to Charity Engine, with your version of BOINC and manually detached each project and then reattached with Charity Engine as the manager. Since seeing my daily production drop to zero on your webpage, I have upgraded BOINC because I was having problems with some workunits hanging at 100% and never reporting as finished. That happened on World Community Grid. When I saw that, I originally tried to reset the project. I still saw workunits hang at or near 100%. SInce upgrading to the newest BOINC, that hasn't happened.
Did I connect my projects to Charity Engine incorrectly?
From Richard's comments, it appears as though WCG jobs are now being processed again, although I'm not seeing any. This raises the question .. when will outstanding credits from WCG jobs completed a couple of years ago be apprlied?
Richard, when did you upgrade your client software? Also, when you say "I have added in Rosetta again" - Do you mean you added this manually, via your desktop client? (*Please note: when you add projects manually, these credits are not applied to CE)
Graham, Richard's issue appears separate from the general issue at WCG, which stems from loss of functionality at WCG servers when that project migrated to Kremble. By way of update, they did impliment a fix for us a couple weeks ago, but we haven't yet done the corresponding work on our end. Apologies for the delay, but hopefull it won't be too much longer on that front.
I am beginning to think I should start over. I had preexisting projects when I started. I may not have allowed CE to set what it needed to do for its projects.
I updated BOINC software within the last week.
Rosetta-
At the time, the project had been in my list. I think when I started CE it had been a project I had but had switched off because they had no work. But you your mention of automatic vs manually adding projects has me concerned I may not have them included them correctly. Is there a way to start over now that they are there, in a way that CE would see them?
Currently, all processors are doing Rosetta. The last time I had WCG work was last Saturday. That was when I refreshed Rosetta and got work from that project.
Richard, if your client is attached to Rosetta using your own BOINC account, it will not show up in your CE stats. If you want to attach to Rosetta using your CE account instead, just detach from Rosetta in the client and it will automatically reattach when it checks in with CE. (You can also trigger this reattachment process manually by choosing "Synchronize with Charity Engine" from the Tools menu).
Now, it looks like there is a CPID mismatch between the accounts that CE creates at projects on your behalf. This makes sense if there were manual attachments and normally it will resolve itself once projects get attached and start earning points through CE. However, the issue you describe with recent points showing as zero makes me think there is something else at play, too. Let's take the first step of detaching Rosetta in the client and letting CE reattach and be sure that has the expected outcome, then we can see if there is a secondary issue to address.
For WCG and Einstien, the button is not grayed out.
Wouldn't that mean that Rosetta is the only one that is attached by CE? (It is. I saw in project properties that both Rosettas were added by the account manager and user id is ce8297942
Maybe I remember a detail I had missed. I had Rosetta ages ago before CE, and they were so sporadic about having work, I stopped using that project. It could be that CE re-established that link to Rosetta while I was thinking they were not reliable for work. I may also have had some work though WCG but when I had issues with some work hanging at 100% and not reporting finished, I reset that project. Could that detach the link to CE?
Edit-
Since I had no work pending for WCG or Einstein on either computer, and I could delete those projects, I did that.
I now have CE and Rosetta on the two computers. Both initially had work. One has run out and is only doing CE work. The other still has Rosetta work to do. I synchronized both computers. I thought maybe the oe that ran out of work might pick up a different project by now, but no.
So, overnight I see no change in my numbers for your website summary. Still zero daily production, and 123,156 total CE.
On the BOINC client, I see CE total now up to 146,367 and daily prod of 1222.36 with Rosetta at daily prod of 881.6
I have been running Rosetta work units on my two computers. I have verified that these two computers with Rosetta are managed by Charaty Engine. I have deleted all other projects that were on my BOINC.
Both computers started out with Rosetta work being processed. One is finishing work units and receiving new work from Rosetta. The other one ran out of Rosetta work units and has been only running CE work.
Is Rosetta the only project available currently? If one of my computers continues to receive Rosetta work, why would the other stop? At first I thought Rosetta didn't have enough work and I would get a new project. At least it is running CE work.
And related to the previous issue for me, the Dashboard on your web page still shows the contest stats at 100% from the last contest completed. And my stats are frozen for my user id of ce8297942 for the last several weeks now. It still shows the CE total production of 123,156 with 0 daily average.
The BOINC client for ce8297942 shows total CE production of 150,832 and average daily production of 1,418.
Richard, above you mentioned that at some point you replaced your CE desktop software with an updated version of BOINC. I wonder if you recall: did you just run the BOINC installer, or did you run any sort of uninstall process prior to that (maybe manually deleting files or folders)?
No. When I upgraded I started with the CE version installed and performed the upgrade using the BOINC installer.
At this point, would it be easier for me to unattach from CE and start from scratch? I see credit going up with CE and one computer is getting Rosetta work currently. But something is off in my original setup I can't find.
Something new to report. Since my one computer had not gotten anything new from Rosetta for a couple of days, I uninstalled BOINC, rebooted my computer, and then reinstalled BOINC. Initially, it didn't appear to have helped anything. But when I rechecked it just now, I have a slew of Rosetta work on it. So, maybe that part is OK. Two computers running Rosetta.
I just have the remaining issue of my stats not updating on your webpage. I see my stats updating on BOINC for ce8297942, but not on your page.
FYI, Credit-awards are not real-time, they are awarded in batches; there are a few layers of latency –
It can take some time (*sometimes days) for credits to be validated and awarded by CE and/or 3rd party research projects. -- Credits will show in your desktop app after this.
Stats at the CE website are updated only once per day, so web stats may lag an additional 24 hrs.
So in your case, if credits just now showed up in your desktop app, please allow 24hours and then check again at the CE website.
Ok, thanks for the updates, Richard. I'm glad the Rosetta work seems to be coming in now. There are two things to cover:
[1] Though the points are not yet showing up on your account dashboard page, I can see that they are being counted in the CE project server behind the scenes. I was thinking this issue with the CPIDs would correct itself, but now see that it needs a bit of a nudge on our end in order for that ball to start rolling. So, I've given it a push and will check back tomorrow to verify that your CPID has been updated correctly first here on this system, then on our stats processing system, and finally at public BOINC projects, as well. It could even take until Friday for this propagation to complete, as it depends on the timing of your computers connecting and also of our daily stats updates.
[2] If you wouldn't mind some additional project attachments, you can opt-in for "Beta Testing" in your account settings and you'll generally get attached to more that way.
Thanks for the update and the extra information on the flow of points. I may opt in as a Beta tester. It sounds interesting.
I have been doing this type of processing since just after Seti@home started. Other than verifying my stats are counting, I am more concerned at this point about my contribution to Charity Engine helping your chosen charities. Just working to collect points can get monotonous after 24 years or so. It is good to know it goes toward more than just crunching the data.
After reviewing some data that one of your computers sent to CE when it last checked in, we can see an usual situation with project attachments. Somehow, Rosetta is attached automatically using an account that isn't a part of CE. In fact, this account was created maybe 3 years before CE even existed. I'm guessing this is left from a different acccount management system, but BOINC is treating it as though CE attached it and thus should be managing it (but it can't).
I think what you can do to correct this is disconnect from Charity Engine from the Tools menu, manually detach the client from all projects, and then reconfigure the client to use CE as the account manager again. This should remove any non-CE attachments and connect only the projects that CE has set up for your account. Then, the different project account CPIDs should merge, stats should be updated, and the correct data should appear on your account dashboard (in due time).
Testing this process on one of your computers should be enough to verify that it works, but it would have to be done on the others, likewise, in order for all of the points to end up in the right place.
We can probably chalk it up to a bit of maintenance being needed after 24 years of BOINC contribution! I'm glad you feel like Charity Engine is an encouragement to continue running in that race and I hope it indeed turns out to be a good fit.
Thanks for that bit on information. I think ages ago I did have a different account manager for a brief time. It was a long enough ago that I forgot it was there. I guess BOINC never forgets. I will check.
Figures shown for the last few days seem to be significantly lower than on previous days, and they don't really correspond with what's shown on the graph. I think the graph figures are correct. Can you check please?
Just to clarify, can you give more specifics on where you're seeing the figures and graphs that don't match? I.e. what doesn't match what?
Here's what I'm seeing today (sent via We Transfer because image upload seems to be broken) ..
https://wetransfer.com/downloads/883b385eebeb86a0600a9522312797d620220601205332/df3f26fa611a1ceca13e7406886dcaad20220601205406/65d06f
Daily average on the dashboard page is shown as: 0
The barchart shows that it should be around 7000.
Thanks for reporting this, Graham. It looks like a stats update process might have gotten stuck following server maintenance. It will be kicking off again soon and we'll know in a day or so whether the problem is fixed. If not, we'll head into debugging round 2.
Round 2 it is... more updates to come as progress is made.
The fix is in. You should now see a giant spike in your chart as more than a week's worth of points have flooded into the latest stats update. Thanks again for reporting.
It was a good opportunity to make a number of updates to the stats processing system that had been on deck!
Something's broken .. again. My Daily Average and Total Points are both shown as zero.
My daily production is way down the last week or so also, and average has dropped also. I thought it was something with my setup.
Hi all. We moved some key backend infrastructure to a new datacenter, which impacted different services to various extents from Sunday to Tuesday. So there's now a bit of a processing backlog, but this should sort itself in the next day or two. No credits or data should be lost. So if you're still seeing missing stats Friday eve or over the weekend, please post back.
My daily average did a one time jump sometime over last weekend. And then dropped in half, and is currently reading zero.
The chart of daily production never increased when the daily average did over the weekend. It appears unchanged and near but above zero.
My computers have been on full time, a total of 16 processors. The only downtime for them occurred over this weekend when Rosetta and World Community Grid both ran dry as far as work.
It was the production before this last weekend that I still wonder about.
So, you said to mention it if your reporting didn't start to update again, last week I think, when the fix was implemented. My stats have not updated since then, all week, here anyway.
From the BOINC, my total production for Charity Engine is 136,655 and my average production is 974.25,
From your webpage, total production fro Charity Engine is 123,156 with 0 daily average, since last week.
So I will just watch that total from the BOINC client instead of the numbers from here.
Richard, can you clarify a few points –
BOINC projects listed are;
Charity Engine
EInstein
Rosetta
World Community Grid
Most recently, I have had the majority of units from World Community Grid. Since they have issues with work running out, I have added in Rosetta again. They also have issues at times. I was hoping between them I would be covered. From my graph of daily work, I had been doing fine, until a bit before I read here that others were having production issues with daily totals dropping to zero. At that time, I had been fine. When I saw that you had said the issue had been fixed, my daily stats started to fall on your page.
When I started Charity Engine, I connected to Charity Engine, with your version of BOINC and manually detached each project and then reattached with Charity Engine as the manager. Since seeing my daily production drop to zero on your webpage, I have upgraded BOINC because I was having problems with some workunits hanging at 100% and never reporting as finished. That happened on World Community Grid. When I saw that, I originally tried to reset the project. I still saw workunits hang at or near 100%. SInce upgrading to the newest BOINC, that hasn't happened.
Did I connect my projects to Charity Engine incorrectly?
From Richard's comments, it appears as though WCG jobs are now being processed again, although I'm not seeing any. This raises the question .. when will outstanding credits from WCG jobs completed a couple of years ago be apprlied?
Richard, when did you upgrade your client software?
Also, when you say "I have added in Rosetta again" - Do you mean you added this manually, via your desktop client?
(*Please note: when you add projects manually, these credits are not applied to CE)
Graham, Richard's issue appears separate from the general issue at WCG, which stems from loss of functionality at WCG servers when that project migrated to Kremble. By way of update, they did impliment a fix for us a couple weeks ago, but we haven't yet done the corresponding work on our end. Apologies for the delay, but hopefull it won't be too much longer on that front.
I am beginning to think I should start over. I had preexisting projects when I started. I may not have allowed CE to set what it needed to do for its projects.
I updated BOINC software within the last week.
Rosetta-
At the time, the project had been in my list. I think when I started CE it had been a project I had but had switched off because they had no work. But you your mention of automatic vs manually adding projects has me concerned I may not have them included them correctly. Is there a way to start over now that they are there, in a way that CE would see them?
Currently, all processors are doing Rosetta. The last time I had WCG work was last Saturday. That was when I refreshed Rosetta and got work from that project.
Richard, if your client is attached to Rosetta using your own BOINC account, it will not show up in your CE stats. If you want to attach to Rosetta using your CE account instead, just detach from Rosetta in the client and it will automatically reattach when it checks in with CE. (You can also trigger this reattachment process manually by choosing "Synchronize with Charity Engine" from the Tools menu).
Now, it looks like there is a CPID mismatch between the accounts that CE creates at projects on your behalf. This makes sense if there were manual attachments and normally it will resolve itself once projects get attached and start earning points through CE. However, the issue you describe with recent points showing as zero makes me think there is something else at play, too. Let's take the first step of detaching Rosetta in the client and letting CE reattach and be sure that has the expected outcome, then we can see if there is a secondary issue to address.
Hmm.
The "Remove" button on Rosetta is grayed out.
For WCG and Einstien, the button is not grayed out.
Wouldn't that mean that Rosetta is the only one that is attached by CE? (It is. I saw in project properties that both Rosettas were added by the account manager and user id is ce8297942
Maybe I remember a detail I had missed. I had Rosetta ages ago before CE, and they were so sporadic about having work, I stopped using that project. It could be that CE re-established that link to Rosetta while I was thinking they were not reliable for work. I may also have had some work though WCG but when I had issues with some work hanging at 100% and not reporting finished, I reset that project. Could that detach the link to CE?
Edit-
Since I had no work pending for WCG or Einstein on either computer, and I could delete those projects, I did that.
I now have CE and Rosetta on the two computers. Both initially had work. One has run out and is only doing CE work. The other still has Rosetta work to do. I synchronized both computers. I thought maybe the oe that ran out of work might pick up a different project by now, but no.
So, overnight I see no change in my numbers for your website summary. Still zero daily production, and 123,156 total CE.
On the BOINC client, I see CE total now up to 146,367 and daily prod of 1222.36 with Rosetta at daily prod of 881.6
Update-
I have been running Rosetta work units on my two computers. I have verified that these two computers with Rosetta are managed by Charaty Engine. I have deleted all other projects that were on my BOINC.
Both computers started out with Rosetta work being processed. One is finishing work units and receiving new work from Rosetta. The other one ran out of Rosetta work units and has been only running CE work.
Is Rosetta the only project available currently? If one of my computers continues to receive Rosetta work, why would the other stop? At first I thought Rosetta didn't have enough work and I would get a new project. At least it is running CE work.
And related to the previous issue for me, the Dashboard on your web page still shows the contest stats at 100% from the last contest completed. And my stats are frozen for my user id of ce8297942 for the last several weeks now. It still shows the CE total production of 123,156 with 0 daily average.
The BOINC client for ce8297942 shows total CE production of 150,832 and average daily production of 1,418.
Richard, above you mentioned that at some point you replaced your CE desktop software with an updated version of BOINC. I wonder if you recall: did you just run the BOINC installer, or did you run any sort of uninstall process prior to that (maybe manually deleting files or folders)?
No. When I upgraded I started with the CE version installed and performed the upgrade using the BOINC installer.
At this point, would it be easier for me to unattach from CE and start from scratch? I see credit going up with CE and one computer is getting Rosetta work currently. But something is off in my original setup I can't find.
Something new to report. Since my one computer had not gotten anything new from Rosetta for a couple of days, I uninstalled BOINC, rebooted my computer, and then reinstalled BOINC. Initially, it didn't appear to have helped anything. But when I rechecked it just now, I have a slew of Rosetta work on it. So, maybe that part is OK. Two computers running Rosetta.
I just have the remaining issue of my stats not updating on your webpage. I see my stats updating on BOINC for ce8297942, but not on your page.
FYI, Credit-awards are not real-time, they are awarded in batches; there are a few layers of latency –
So in your case, if credits just now showed up in your desktop app, please allow 24hours and then check again at the CE website.
Thanks. I figured there must be a validation process.
Maybe tomorrow the total will update.
Thanks for sticking with this issue for me.
Ok, thanks for the updates, Richard. I'm glad the Rosetta work seems to be coming in now. There are two things to cover:
[1] Though the points are not yet showing up on your account dashboard page, I can see that they are being counted in the CE project server behind the scenes. I was thinking this issue with the CPIDs would correct itself, but now see that it needs a bit of a nudge on our end in order for that ball to start rolling. So, I've given it a push and will check back tomorrow to verify that your CPID has been updated correctly first here on this system, then on our stats processing system, and finally at public BOINC projects, as well. It could even take until Friday for this propagation to complete, as it depends on the timing of your computers connecting and also of our daily stats updates.
[2] If you wouldn't mind some additional project attachments, you can opt-in for "Beta Testing" in your account settings and you'll generally get attached to more that way.
More to come in the next 24-48 hours.
Thanks for the update and the extra information on the flow of points. I may opt in as a Beta tester. It sounds interesting.
I have been doing this type of processing since just after Seti@home started. Other than verifying my stats are counting, I am more concerned at this point about my contribution to Charity Engine helping your chosen charities. Just working to collect points can get monotonous after 24 years or so. It is good to know it goes toward more than just crunching the data.
After reviewing some data that one of your computers sent to CE when it last checked in, we can see an usual situation with project attachments. Somehow, Rosetta is attached automatically using an account that isn't a part of CE. In fact, this account was created maybe 3 years before CE even existed. I'm guessing this is left from a different acccount management system, but BOINC is treating it as though CE attached it and thus should be managing it (but it can't).
I think what you can do to correct this is disconnect from Charity Engine from the Tools menu, manually detach the client from all projects, and then reconfigure the client to use CE as the account manager again. This should remove any non-CE attachments and connect only the projects that CE has set up for your account. Then, the different project account CPIDs should merge, stats should be updated, and the correct data should appear on your account dashboard (in due time).
Testing this process on one of your computers should be enough to verify that it works, but it would have to be done on the others, likewise, in order for all of the points to end up in the right place.
We can probably chalk it up to a bit of maintenance being needed after 24 years of BOINC contribution! I'm glad you feel like Charity Engine is an encouragement to continue running in that race and I hope it indeed turns out to be a good fit.
Thanks for that bit on information. I think ages ago I did have a different account manager for a brief time. It was a long enough ago that I forgot it was there. I guess BOINC never forgets. I will check.
I refreshed the Charity Engine management on all three computers. The old Rosetta connection is gone and replaced by CE.
And today I see the first production numbers in a long time, coming through on my CE Dashboard. Everything looks good on my end.
Thanks for your help.
(*Graham - Work and further communications have been ongoing with WCG, so hopefully we'll have some constructive progress on that issue soon....)