Note

You can download this example as a Jupyter notebook or start it in interactive mode.

Remote Solving with SSH

This example demonstrates how linopy can solve optimization models on remote machines using SSH connections. This is one of two remote solving options available in linopy 1. SSH Remote Solving (this example) - Connect to your own servers via SSH:nbsphinx-math:n2. OETC Cloud Solving - Use cloud-based optimization services (see solve-on-oetc.ipynb)

SSH Remote Solving:nbsphinx-math:n\nSSH `remote solving is ideal when you have::nbsphinx-math:n* Access to dedicated servers with optimization solvers installed:nbsphinx-math:n* Full control over the computing environment:nbsphinx-math:n* Existing infrastructure for optimization workloads:nbsphinx-math:n`:nbsphinx-math:n`## What you need for SSH remote solving::nbsphinx-math:n* A running installation of paramiko on your local machine (``pip install paramiko`):nbsphinx-math:n* `A remote server with a working installation of linopy (e.g., in a conda environment):nbsphinx-math:n* SSH access to that machine:nbsphinx-math:n`:nbsphinx-math:n`## How SSH Remote Solving Works:nbsphinx-math:n`:nbsphinx-math:nThe `workflow consists of the following steps, most of which linopy handles automatically::nbsphinx-math:n`:nbsphinx-math:n1. Define a model on the local machine:nbsphinx-math:n2. Save the model on the remote machine via SSH:nbsphinx-math:n3. Load, solve and write out the model on the remote machine:nbsphinx-math:n4. Copy the solved model back to the local machine:nbsphinx-math:n5. Load the solved model on the local machine:nbsphinx-math:n:nbsphinx-math:`nThe `model initialization happens locally, while the actual solving happens remotely.”

Note: This notebook requires SSH access to a remote server with a solver installed. It is not executed during the documentation build, so no cell outputs are shown. To run it yourself, configure SSH access and install a solver on the remote machine.

Create a model

First we are going to build the optimization model we want to solve in our local process.

from numpy import arange
from xarray import DataArray

from linopy import Model

N = 10
m = Model()
coords = [arange(N), arange(N)]
x = m.add_variables(coords=coords, name="x")
y = m.add_variables(coords=coords, name="y")
m.add_constraints(x - y >= DataArray(arange(N)))
m.add_constraints(x + y >= 0)
m.add_objective((2 * x + y).sum())
m

Linopy LP model
===============

Variables:
----------
 * x (dim_0, dim_1)
 * y (dim_0, dim_1)

Constraints:
------------
 * con0 (dim_0, dim_1)
 * con1 (dim_0, dim_1)

Status:
-------
initialized

Initialize SSH connection

Now we have to set up the SSH connection. The SSH connection is handled by the RemoteHandler class in of the linopy.remote module. This is strongly relying on the paramiko package. When initializing, you have two options:

1. Pass the standard arguments host, username. If the SSH keys are stored in a default location, the keys are autodetected and the RemoteHandler does not require the password argument. Otherwise you also have to pass the password.

2. Pass a working paramiko.SSHClient as client. This enables you to set up the SSH connection by others means supported by paramiko.

In the following we use the first option.

from linopy import RemoteHandler

host = "your.host.de"
username = "username"

handler = RemoteHandler(host, username=username)

Optionally: Activate a conda environment on the remote

The RemoteHandler keeps an interactive shell in the background. You can execute any code in order to prepare the solving process (install linopy, activate an environment).

Assuming you have a conda environment linopy-env that contains the linopy package with dependencies, you can run

handler.execute("conda activate linopy-env")

Solve the model on remote

Now the only thing you have to do is to pass the RemoteHandler as an argument to the solve function. Other keyword arguments like solver_name and solver options are propagated to the remote machine.

m.solve(remote=handler)

Set parameter Username
Academic license - for non-commercial use only - expires 2023-02-06
Read LP format model from file /tmp/linopy-problem-uh4gvjyp.lp
Reading time = 0.00 seconds
obj: 200 rows, 200 columns, 400 nonzeros
Gurobi Optimizer version 9.5.1 build v9.5.1rc2 (linux64)
Thread count: 12 physical cores, 24 logical processors, using up to 24 threads
Optimize a model with 200 rows, 200 columns and 400 nonzeros
Model fingerprint: 0xf2bcac49
Coefficient statistics:
Matrix range     [1e+00, 1e+00]
Objective range  [1e+00, 2e+00]
Bounds range     [0e+00, 0e+00]
RHS range        [1e+00, 9e+00]
Presolve removed 200 rows and 200 columns
Presolve time: 0.00s
Presolve: All rows and columns removed
Iteration    Objective       Primal Inf.    Dual Inf.      Time
0    2.2500000e+02   0.000000e+00   0.000000e+00      0s

Solved in 0 iterations and 0.00 seconds (0.00 work units)
Optimal objective  2.250000000e+02

('ok', '')

m.solution

<xarray.Dataset>
Dimensions:  (dim_0: 10, dim_1: 10)
Coordinates:
  * dim_0    (dim_0) int64 0 1 2 3 4 5 6 7 8 9
  * dim_1    (dim_1) int64 0 1 2 3 4 5 6 7 8 9
Data variables:
    x        (dim_0, dim_1) float64 0.0 0.0 0.0 0.0 0.0 ... 4.5 4.5 4.5 4.5 4.5
    y        (dim_0, dim_1) float64 0.0 0.0 0.0 0.0 0.0 ... -4.5 -4.5 -4.5 -4.5

xarray.Dataset

• Dimensions:
  • dim_0: 10
  • dim_1: 10
• Coordinates: (2)
  • dim_0
    (dim_0)
    int64
    0 1 2 3 4 5 6 7 8 9

    array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

  • dim_1
    (dim_1)
    int64
    0 1 2 3 4 5 6 7 8 9

    array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

• Data variables: (2)
  • x
    (dim_0, dim_1)
    float64
    0.0 0.0 0.0 0.0 ... 4.5 4.5 4.5 4.5

    array([[0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. ],
           [0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5],
           [1. , 1. , 1. , 1. , 1. , 1. , 1. , 1. , 1. , 1. ],
           [1.5, 1.5, 1.5, 1.5, 1.5, 1.5, 1.5, 1.5, 1.5, 1.5],
           [2. , 2. , 2. , 2. , 2. , 2. , 2. , 2. , 2. , 2. ],
           [2.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5, 2.5],
           [3. , 3. , 3. , 3. , 3. , 3. , 3. , 3. , 3. , 3. ],
           [3.5, 3.5, 3.5, 3.5, 3.5, 3.5, 3.5, 3.5, 3.5, 3.5],
           [4. , 4. , 4. , 4. , 4. , 4. , 4. , 4. , 4. , 4. ],
           [4.5, 4.5, 4.5, 4.5, 4.5, 4.5, 4.5, 4.5, 4.5, 4.5]])

  • y
    (dim_0, dim_1)
    float64
    0.0 0.0 0.0 0.0 ... -4.5 -4.5 -4.5

    array([[ 0. ,  0. ,  0. ,  0. ,  0. ,  0. ,  0. ,  0. ,  0. ,  0. ],
           [-0.5, -0.5, -0.5, -0.5, -0.5, -0.5, -0.5, -0.5, -0.5, -0.5],
           [-1. , -1. , -1. , -1. , -1. , -1. , -1. , -1. , -1. , -1. ],
           [-1.5, -1.5, -1.5, -1.5, -1.5, -1.5, -1.5, -1.5, -1.5, -1.5],
           [-2. , -2. , -2. , -2. , -2. , -2. , -2. , -2. , -2. , -2. ],
           [-2.5, -2.5, -2.5, -2.5, -2.5, -2.5, -2.5, -2.5, -2.5, -2.5],
           [-3. , -3. , -3. , -3. , -3. , -3. , -3. , -3. , -3. , -3. ],
           [-3.5, -3.5, -3.5, -3.5, -3.5, -3.5, -3.5, -3.5, -3.5, -3.5],
           [-4. , -4. , -4. , -4. , -4. , -4. , -4. , -4. , -4. , -4. ],
           [-4.5, -4.5, -4.5, -4.5, -4.5, -4.5, -4.5, -4.5, -4.5, -4.5]])

• Attributes: (0)