I read over at mersenneforum.org that people with a Google account have access to free GPUs (Tesla K80) at https://colab.research.google.com/. People at mersenneforum.org wrote a python script that can run the trial factoring programm mfactc for Mersenne numbers, so I wondered if Colab can also be used for PG apps. I tweaked the script a bit and was able to run the GFN sieve program and Genefer (standalone).
It works like this: You upload the program you want to run to your Google Drive in a folder, then run the program from a python script in Colab (which you give permission to access your Drive files). The results get saved in your Google Drive.

Quick Tutorial for MSR:
1. Have a folder called "gfn_sieve" in your Google Drive with "gfnsvocl_linux_x86_64" in it.
2. Get this python script and add it to Colab: https://colab.research.google.com/drive/1RC_-Rjpxw4SFAFuXImQMRNm6UlYpHuIt
3. Tweak the last line: "... /usr/local/bin/gfnsvocl_linux_x86_64 23 156399 156400" to the range you want to run.
4. Run (play button in the upper left).
5. Leave the browser window open. If anything breaks, try "Runtime -> Reset all runtimes..." and start again. GFN sieve has checkpoints, so you can continue at any point.

Genefer is run similarly (I have a script for those interested), but I'm not sure how to get it run with BOINC. If someone knows how to integrate BOINC to Colab it would be great, else could we use it for manual GFN double checking or something like that?

Anyway... for those into MSR, have fun with some free GPUs ;)

1. Have a folder called "gfn_sieve" in your Google Drive with "gfnsvocl_linux_x86_64" in it.

How do you get "gfnsvocl_linux_x86_64" in it?

This keeps repeatedly coming up:
Error obtaining number of platforms (clGetPlatformIDs/1)
Error code -1001, message: Unknown
No compatible OpenCL devices found

What I've observed over the last couple of days is that Google Colab gives out packages of "1 GPU - 12 hours runtime" before you have to restart the script. Here is a cheap trick i tried out the last days: If you copy the sieve script (with a different command line) and use a second folder, you can run two GPU tasks at once in two browser tabs. However, assuming you start them at the same time, both will stop after 6 hours with an error message that no host with GPU could be found (and after 4 hours if you use 3 scripts etc.). After another 6 hours (or 8 hours in case of 3 scripts) host can be found again. So it seems that with this trick you don't get more computational resources but rather you can only use up your computational "budget" faster.

HI
@matzetoni
Thanks to you we have great "fun" :)
Regarding the 12h limit, how many pauses do you have to do to add a bud before this period?
I wish you success and script development
Pozdr.
Tomasz Kaczmarek
____________

If one runs multiple scripts at the same time and pause the scripts now and then, it is hard to predict when exactly Colab decides to give GPUs to you or take GPUs from you. Overall my observation is that it does not give you any advantage compared to running only one script 12 h straight (even worse: you have to organize more scripts). Of course no one stops you from creating a second Google account...

Thanks to user Dylan14 at mersenneforum, we can run boinc / PG on Google Colab using the following script. There is no need of Google Drive here, so you can just copy the script, insert your account key and run it.

For a GPU enabled host I get:
PPSE ~ 1h
GFN16 ~ 3 min 30 sec

import os.path !apt-get update !apt-get install boinc boinc-client !cp /usr/bin/boinc /content !cp /usr/bin/boinccmd /content if not os.path.exists('/content/slots'): !mkdir slots if not os.path.exists('/content/slots/0'): !boinc --daemon --attach_project "http://www.primegrid.com/" "account key here" else: !boinc --daemon !boinccmd --set_gpu_mode always !tail -f stdoutdae.txt

This helped me so the GPU wasn't timing out all the time :)

At least I don't know one. This script works entirely without Google Drive. If you mount your Drive, you could save your checkpoints. But part of the problem with continuing your work is that when you start a new Colab session you don't necessarily get the same instance you had in the previous session. And BOINC does not simply continue a task on a new host that was originally sent out to another host. Because tasks that get lost when your instance aborts are only resent after timeout, I would recommend using this script only for short tasks, mainly GFN15 / GFN16, where the timeout period is short.

But part of the problem with continuing your work is that when you start a new Colab session you don't necessarily get the same instance you had in the previous session.

I played with this briefly but BOINC did not detect any GPU, only two CPUs. Did I miss anything?

D'oh!! I didn't even notice that option.

For anyone else having this problem, or just not paying attention like me, the setting is at the upper left of the page.

Thanks.

I am finding that the environment is stopping after a while. I think it may have been because I had set the "Maintain enough work for an additional" parameter to 0.00001 days. I have put it back to 0.0005 days and I shall see how that goes.

At least I don't know one. This script works entirely without Google Drive. If you mount your Drive, you could save your checkpoints. But part of the problem with continuing your work is that when you start a new Colab session you don't necessarily get the same instance you had in the previous session. And BOINC does not simply continue a task on a new host that was originally sent out to another host. Because tasks that get lost when your instance aborts are only resent after timeout, I would recommend using this script only for short tasks, mainly GFN15 / GFN16, where the timeout period is short.

It's not working for me any more. Not sure why. The latest run managed to complete 5 WUs before dieing and lasted something like an hour.

It's not working for me any more. Not sure why. The latest run managed to complete 5 WUs before dieing and lasted something like an hour.

It's not working for me any more. Not sure why. The latest run managed to complete 5 WUs before dieing and lasted something like an hour.

It still works for me.

import os.path !apt-get update !apt-get install boinc boinc-client !cp /usr/bin/boinc /content !cp /usr/bin/boinccmd /content if not os.path.exists('/content/slots'): !mkdir slots if not os.path.exists('/content/slots/0'): !boinc --attach_project "http://www.primegrid.com/" "account key here" else: !boinc

I noticed the terms of service for colab say no cryptocurrency mining.

https://research.google.com/colaboratory/faq.html

Please note that using Colaboratory for cryptocurrency mining is disallowed entirely, and may result in being banned from using Colab altogether.

I'm not sure they'll like people using BOINC either.

I have only done the occasional GFN Sieve on Google Colab as I don't want to abuse the free platform. I noticed today when I just started a sieve that the GPU type was different. It wasn't showing the Tesla K80, but a P100.

D0: "Tesla P100-PCIE-16GB"

I noticed the terms of service for colab say no cryptocurrency mining.

https://research.google.com/colaboratory/faq.html

Please note that using Colaboratory for cryptocurrency mining is disallowed entirely, and may result in being banned from using Colab altogether.

I'm not sure they'll like people using BOINC either.

I too have begun to get Tesla P100 16GB GPU's on Google Colab.

I was able to do 100P of GFN21 in the allotted 12hr window.

It's important to note that you don't want to run these constantly. I had some issues with the P80's where after a few consecutive sessions back to back for a few days I'd get a connection error of no GPU or no backend available. This is Colab limiting high intense computing. I haven't tried to reach my limit with the P100's but I imagine it's the same deal. If you run them constantly you get flagged for high intense use and they limit you for a brief time of a few days. I took a week or so off and went back to and had no more issues but was lucky enough to now get P100's which seem to be nearly 3x as fast.

I just signed up today for their free 750 hrs/month. I have one instance running (1 vCPU only) and will see how things go.

Has anyone tried to see how many such 1vCPU instances we can request? I went through a mock request for a second such instance (1 vCPU, 750 hrs/month) and it is not objecting.

I too have begun to get Tesla P100 16GB GPU's on Google Colab.

I am not getting a GPU at all any more.

Still working for me, although the last 2 runs (weekend before last and today) gave me Tesla K80's.

Had you been using the service a lot? I think someone mentioned further up about restrictions for repeated high usage.

It has been a few days since your post, so maybe try again if you haven't in the meantime.

Still working for me, although the last 2 runs (weekend before last and today) gave me Tesla K80's.

Had you been using the service a lot? I think someone mentioned further up about restrictions for repeated high usage.

It has been a few days since your post, so maybe try again if you haven't in the meantime.

I saw a T4 GPU one time on Colab. Only K80's and P100's every other time. Specs for the T4 put it similar to a 2070 Super in terms of CUDA cores. I did not benchmark it properly but I imagine it could easily do 250-300P in the allotted 12hrs.

One time only, and have not gotten one since.

I successully ran a GFN21 task in about five hours on a P100. This is faster than a 2080 Ti; evidently those 4 TeraFLOPS of DP compute flexing their muscles. As far as I know, GFN21 and 22 (not DYFL) are the only projects that can still make use of DP on account of the OCL transform.

Anyway, could be useful for the upcoming challenge. If you can get a P100. :)

import os.path !apt-get update !apt-get install boinc boinc-client !cp /usr/bin/boinc /content !cp /usr/bin/boinccmd /content if not os.path.exists('/content/slots'): !mkdir slots if not os.path.exists('/content/slots/0'): !boinc --daemon --attach_project "http://www.primegrid.com/" "account key here" else: !boinc --daemon !boinccmd --set_gpu_mode always !tail -f stdoutdae.txt

This helped me so the GPU wasn't timing out all the time :)

I saw a T4 GPU one time on Colab. Only K80's and P100's every other time. Specs for the T4 put it similar to a 2070 Super in terms of CUDA cores. I did not benchmark it properly but I imagine it could easily do 250-300P in the allotted 12hrs.

One time only, and have not gotten one since.

Genefer 20 on Tesla P100, run time 7836 seconds
https://www.primegrid.com/result.php?resultid=1053705624
Genefer 20 on Tesla T4, run time 13133 seconds
https://www.primegrid.com/result.php?resultid=1054157647

https://www.primegrid.com/workunit.php?wuid=644461755
A WU with both tasks on Google Colab.
GFN 17 low, less than 5 minutes on T4, and more than 10 minutes on P4.

I have been using colab. figured I would post up some numbers based on the different gpus I get. I have been able to get a single instance 12 hours a day or so and then a break of 12 hours. non gpu instances are available during that time, but I get timeouts faster, so don't bother.

ppssieve.
p100 216 seconds
t4 206 seconds
p4 388 seconds

ap27
p100 1063 seconds
t4 1079 seconds
p4 1781 seconds

gfn15
p100 36 seconds
t4 27 seconds
p4 58 seconds
k80 56 seconds

Have been using the Google Colab on and off for about a month now to run PrimeGrid and the last week it automatically "times out" with message it is stopped "due to Inactivity" after about one to two hours.

I know it's free to use... was wondering if there is anything I can do to keep it up and running for more than a hour?

On a related question how can I also attach my WuProp account along with PrimeGrid if I can get it to stay active for more than 6 hours at a time?
____________

Have been using the Google Colab on and off for about a month now to run PrimeGrid and the last week it automatically "times out" with message it is stopped "due to Inactivity" after about one to two hours.

I know it's free to use... was wondering if there is anything I can do to keep it up and running for more than a hour?

If I had to guess why it's doing this now, it's that Google is cheaping out on many of their giveaways. I'm involved with Startup Weekend, and they reduced the amount of free cloud computing from $3000 per person to $300. Their advertising business has taken a huge hit during the pandemic, and they were already trying to figure out how to stop hemorrhaging money.

My adjusted code:

!nvidia-smi -L
!lscpu | grep MHz
import os.path
!apt-get update > /dev/null 2>&1
!hostname 'Colab'
!bash -c -e "echo 'Colab' > /etc/hostname"
!bash -c -e "echo '127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters 172.28.0.2 Colab' > /etc/hosts"
!apt-get install boinc boinc-client > /dev/null 2>&1
!cp /usr/bin/boinc /content
!cp /usr/bin/boinccmd /content
!bash -c -e "echo '<cc_config> <log_flags> <task>0</task> <file_xfer>0</file_xfer> <sched_ops>1</sched_ops> <rr_simulation>0</rr_simulation> </log_flags> <options> <abort_jobs_on_exit>0</abort_jobs_on_exit> <use_all_gpus>1</use_all_gpus> <allow_multiple_clients>0</allow_multiple_clients> <max_file_xfers_per_project>20</max_file_xfers_per_project> <max_file_xfers>30</max_file_xfers> <max_tasks_reported>1000</max_tasks_reported> <process_priority>5</process_priority> <process_priority_special>5</process_priority_special> <report_results_immediately>1</report_results_immediately> </options> </cc_config>' > /content/cc_config.xml"
if not os.path.exists('/content/slots'):
!mkdir slots
if not os.path.exists('/content/slots/0'):
!timeout 55 boinc --attach_project "http://boincvm.proxyma.ru:30080/test4vm/" "WEAK_ACCOUNT_KEY"
!bash -c -e "echo '<app_config> <app> <name>gfn13</name> <report_results_immediately/> </app> <app_version> <app_name>gfn13</app_name> <plan_class>opencl_nvidia_gfn_101</plan_class> <avg_ncpus>1</avg_ncpus> <ngpus>0.2</ngpus> <cmdline>-x ocl4</cmdline> </app_version> <app> <name>gfn14</name> <report_results_immediately/> </app> <app_version> <app_name>gfn14</app_name> <plan_class>opencl_nvidia_gfn_101</plan_class> <avg_ncpus>1</avg_ncpus> <ngpus>0.2</ngpus> <cmdline>-x ocl4</cmdline> </app_version> </app_config>' > /content/projects/boincvm.proxyma.ru_30080_test4vm/app_config.xml"
!boinc
else:
!boinc


If you want to close your instance please do the following code (this abort all tasks that are there and thus they can't get in limbo). There is a lot of repetition here but for some reason it doesn't abort properly at the first try.:
!pkill boinc > /dev/null 2>&1
!bash -c -e "echo '<cc_config> <log_flags> <task>0</task> <file_xfer>0</file_xfer> <sched_ops>1</sched_ops> <rr_simulation>0</rr_simulation> </log_flags> <options> <abort_jobs_on_exit>1</abort_jobs_on_exit> <use_all_gpus>1</use_all_gpus> <allow_multiple_clients>0</allow_multiple_clients> <max_file_xfers_per_project>20</max_file_xfers_per_project> <max_file_xfers>30</max_file_xfers> <max_tasks_reported>1000</max_tasks_reported> <process_priority>5</process_priority> <process_priority_special>5</process_priority_special> <report_results_immediately>1</report_results_immediately> </options> </cc_config>' > /content/cc_config.xml"
!timeout 4 boinc > /dev/null 2>&1
!pkill boinc > /dev/null 2>&1
!pkill boinc > /dev/null 2>&1
!bash -c -e "echo '<cc_config> <log_flags> <task>0</task> <file_xfer>0</file_xfer> <sched_ops>1</sched_ops> <rr_simulation>0</rr_simulation> </log_flags> <options> <abort_jobs_on_exit>1</abort_jobs_on_exit> <use_all_gpus>1</use_all_gpus> <allow_multiple_clients>0</allow_multiple_clients> <max_file_xfers_per_project>20</max_file_xfers_per_project> <max_file_xfers>30</max_file_xfers> <max_tasks_reported>1000</max_tasks_reported> <process_priority>5</process_priority> <process_priority_special>5</process_priority_special> <report_results_immediately>1</report_results_immediately> </options> </cc_config>' > /content/cc_config.xml"
!timeout 4 boinc > /dev/null 2>&1
!pkill boinc > /dev/null 2>&1


If you get connection issues with colab please use this javascript in your browser (CTRL+SHIRT+I) (this presses the connect button every 60 seconds):
function ConnectButton(){
console.log("Connect pushed");
document.querySelector("#top-toolbar > colab-connect-button").shadowRoot.querySelector("#connect").click()
}
setInterval(ConnectButton,60000);

On a related question how can I also attach my WuProp account along with PrimeGrid if I can get it to stay active for more than 6 hours at a time?

mattozan,

Thanks for the reply. I had tried the same "tricks" as you for a second project with same ineffective results on CoLab.

I must of been very lucky... just had a free CoLab session with GPU run for about 12 hours after not using it for almost a month.
____________

Just add projects one after the other, for example:

!boinccmd --project_attach "http://www.primegrid.com/" "your_pg_key" !boinccmd --project_attach "https://wuprop.boinc-af.org/" "your_wu_key"

Can someone please spoon-feed me how to get AP27 running or GFN17? The initial instruction are not clear enough for me.

i wise i had to skills to do this.