I know people run multiple WUs on 1 GPU, but can it be done the other way round? Primegrid produces huge tasks that take days on a GPU. Can they be spread across more than one GPU, as a CPU workunit can go across 2 CPUs on a server MB? If the GPUs are connected with SLI/crossfire, will Boinc treat them as one?

I know people run multiple WUs on 1 GPU, but can it be done the other way round? Primegrid produces huge tasks that take days on a GPU. Can they be spread across more than one GPU, as a CPU workunit can go across 2 CPUs on a server MB? If the GPUs are connected with SLI/crossfire, will Boinc treat them as one?

No you can not, Boinc is not setup for that and the Developers haven't publicly said they were interested in doing it either. You might send a note to Richard Haselgrove asking him if the Developers are even thinking about adding it as he's the head of the group. I don't know if he's on here at Prime Grid or not though.

Theoretically, yes.

But the app would have to be written specifically for it, and I’m not aware of any app anywhere that does this. “Why not?”, you might ask. It’s because there’s absolutely no benefit to doing this, while it makes the apps far more complex, and more costly to develop and more costly to maintain. If a task takes an hour on one GPU and half an hour on two GPUs, it doesn’t matter if you run one task (on both GPUs) or two tasks (each on one GPU) ; either way you’re doing two tasks per hour. There’s no benefit here, and your apps are more complex.

If you’re playing a video game, SLI makes sense. Doubling the frame rate can make a huge difference in playability. But that’s not the case with crunching.

Not only would using SLI not be helpful, but you literally can not use it for crunching. If you enable SLI, the apps can’t see the GPUs at all and can’t run.

The main problem is the memory which is intensively used.
With CPUs, multithreading is efficient if all threads run on a single processor and that the shared memory is the L3 cache of the processor. If the threads are executed on two processors, the shared area is the main memory and multithreading is not efficient.
Two GPUs would have to exchange data continuously and the bandwith of PCIe is very low compared to GPU internal memory bus.

Once you turn on SLI, CUDA (and presumably OpenCL) can't see the GPUs at all.

Moo! runs a single task across multiple GPUs. They do RC5-72 for distributed.net, in a BOINC wrapper.

Not only would using SLI not be helpful, but you literally can not use it for crunching. If you enable SLI, the apps can’t see the GPUs at all and can’t run.

Ah, I assumed when you enabled SLI, that the game thought you had a GPU with more cores in it. I didn't realise the game had to do anything special to make use of it.

You misunderstand.

Games are different than crunching apps.

SLI is meant for games. It lets the two GPUs work together so you get a better gaming experience.

However, games and crunching apps use completely different interfaces/APIs to communicate with the GPU. The game's interface is for drawing stuff on the screen, while our apps use an interface that doesn't draw on the screen. It's only used for doing calculations.

The interface we use gets turned off when SLI is enabled, so it's literally impossible to use SLI for crunching.

An app can still use more than one GPU, but it can't use SLI, so the GPUs can't share data directly. This limits its usefulness for some/many/most applications. And as I said originally, you would have to write the app specifically to use multiple GPUs.

I'm not a video game programmer, so I don't know if you have to write games specifically to make use of SLI or if it just "works".

Moo! runs a single task across multiple GPUs. They do RC5-72 for distributed.net, in a BOINC wrapper.

Has this been specifically written to do so then? It seems Boinc doesn't have this ability built in. What do you see in Boinc Manager / Boinctasks when you do this?

Standard BOINC client. Not sure what was done inside the wrapper of the task. Here are a couple of screen shots, from both boincmgr and boinctasks.

Not only would using SLI not be helpful, but you literally can not use it for crunching. If you enable SLI, the apps can’t see the GPUs at all and can’t run.

Ah, I assumed when you enabled SLI, that the game thought you had a GPU with more cores in it. I didn't realise the game had to do anything special to make use of it.

You misunderstand.

Games are different than crunching apps.

SLI is meant for games. It lets the two GPUs work together so you get a better gaming experience.

However, games and crunching apps use completely different interfaces/APIs to communicate with the GPU. The game's interface is for drawing stuff on the screen, while our apps use an interface that doesn't draw on the screen. It's only used for doing calculations.

The interface we use gets turned off when SLI is enabled, so it's literally impossible to use SLI for crunching.

An app can still use more than one GPU, but it can't use SLI, so the GPUs can't share data directly. This limits its usefulness for some/many/most applications. And as I said originally, you would have to write the app specifically to use multiple GPUs.

I'm not a video game programmer, so I don't know if you have to write games specifically to make use of SLI or if it just "works".

I assumed that at some point the game does some physics, to calculate your character jumping and falling, the trajectory of a bullet, etc. And that uses the same stuff as Boinc? Or does the game access each GPU individually for that part?

What part of "I'm not a video game programmer" isn't clear?

To be clear, I was not advocating that PG should do this. Only that it was technically possible using BOINC.

Moo!Wrapper makes use of the distributed.net client within the BOINC wrapper.
The dnetc client can divide the multiple packets of work between multiple GPUs, just like the dnetc client can divide the work between multiple CPUs - and can in fact mix GPUs and CPUs, but Moo!Wrapper doesn't make use of this (quite sensibly IMHO).
The BOINC client only knows that it uses multiple GPUs for scheduling purposes - it handles the task as a single process.
In summary, the dnetc client is a complete and fully featured client, rather than just a wrapper.

So expecting BOINC to manage this generically is probably not realistic.
And expecting other project wrappers to implement this is also not realistic IMHO.
Especially since it would be an almost identical outcome to just run 2x single GPU tasks and BOINC and the project apps can already do this.
A lot of effort and complexity for virtually no gain.

FWIW, here is the stderr output from a moo tasks running on two GPUs. Not sure how long it will stick around, so look quickly.

https://moowrap.net/result.php?resultid=124376635

Moo! sends you a packet containing few real distributed.net client tasks.

The distributed.client can use multiple GPUs in single instance, it's not difficult - each GPU is handled by own thread/subprocess. But each GPU is running its own, independent task from input buffer.

Note that at end of the input buffer client have to wait for slowest GPU to finish it's task, and another GPU is idle. So this is a not a most efficient way to run things.

The only gain I was thinking of was making the huge genefer extreme tasks finish in a more reasonable time. But I guess they're not in a hurry like some Biology projects are.

Everything that increases total throughput is certainly welcome on PG, I guess.

But in GPU case it might run one task quicker, but unless it takes less than 50% of the time a single-GPU task takes, it will not increase throughput. If you finish a GFN extreme task on one GPU in 3 days, you finish 2 tasks every three days on your 2-GPU-system. If running one task on two GPUs simultaneously makes it finish in 1.5 days you still finish 2 tasks every three days.

The only reason why many LLR2 tasks here are run multithreaded is due to limited L3 cache as far as I know. Otherwise single-threading is almost always faster in terms of total throughput.

To be clear, I was not advocating that PG should do this. Only that it was technically possible using BOINC.

Are both gpu's the same in the machine?

In the one I sent the link to? Yes. Both 1660 Tis. But they don't have to be the same at Moo!