Knowledge Base

I want to use GPUs for running Desmond. How do I configure the machine and the software?

Schrödinger has partnered with Exxact Corporation to provide pre-configured and pre-installed machines for running Desmond on GPUs. See the MD Compatible Systems page for more information.

To configure your own machine, follow the instructions below.

First, check that hardware and software meet the requirements:

  • Check for the supported Graphics cards, drivers, and operating systems on the Supported Platforms page.
  • The total memory of all cards used must be sufficient to run the simulations. A rule of thumb is about 1 GB per 100,000 atoms. For FEP you can use multiple cards for a simulation, but for standard molecular dynamics simulations, only one card can be used per simulation.
  • Ensure that your machine has adequate cooling for the graphics cards. Running simulations on a GPU generates a lot of heat which must be dissipated. Inadequate cooling may lead to lower performance and damages of the graphics cards.

Second, ensure that all your cards are configured properly. You need administrative permissions to run the following commands.

  1. Disable ECC memory:
    nvidia-smi -e 0
  2. Reboot the machines. Check the output of nvidia-smi to ensure all GPUs have ECC memory disabled.
  3. Set the compute Mode of the GPU to "Exclusive Process":
    nvidia-smi -c 3

    On some systems, you might need to set the compute mode for each card individually:

    • nvidia-smi -c 3 -i 0
    • nvidia-smi -c 3 -i 1
    • ...

    To enable this options automatically, add the above lines to /etc/rc.local.

  4. To allow a faster startup of the CUDA runtime, set persistence mode:
    nvidia-smi -pm 1

    This line can also be added to /etc/rc.local to ensure it is set on boot.

  5. Ensure that /dev/nvidiactl is created and the kernel module is loaded. Add the line below to /etc/rc.local as well.
    modprobe nvidia
  6. Ensure that users can read and write to the /dev/nvidia* devices:
    chmod 666 /dev/nvidia*

Next, set up and configure the software.

  1. Install the Schrödinger software.
  2. Ensure that the GPU cards are recognized by Schrödinger Software, by running the following command:
    $SCHRODINGER/utilities/query_gpgpu -a
  3. Using GPUs in combination with FEP jobs requires a batch queueing system. For other jobs it is highly recommended.
  4. Configure the schrodinger.hosts file for GPUs as described in KB article 1844.

Important Notes

  • Placing the GPU in "Exclusive Process" mode allows only one compute process at a time. This does not affect graphical applications, such as Maestro
  • It is recommended to use a separate graphics card for connecting to a monitor reserving other GPUs for compute processes only. Sharing a GPU for both display and calculations may reduce the lifetime of the GPU.

Contact Technical Support if you need help setting up the hosts file for batch queues (e.g. clusters) that manage hosts with GPUs.

To ask a question or get help, please submit a support ticket or email us at help@schrodinger.com.

Back To Top