Skip to main content

ROFL Node

info

These instructions are for setting up a ROFL node which executes ROFLs inside a TEE, but otherwise only observes the ParaTime activity and can also submit transactions.

This guide will cover setting up your ROFL node for running ROFL apps on the Oasis Network. This guide assumes some basic knowledge on the use of command line tools.

Prerequisites​

ParaTime Client Node​

ROFL node is a special kind of a ParaTime Client Node with a TEE-capable hardware in order to securely run ROFLs. First, complete the ParaTime Client Node instructions:

  1. Set up your ParaTime Client Node including support for TEE ParaTimes.
  2. Add the Sapphire ParaTime to your node (Mainnet, Testnet).
  3. To support ROFL TDX apps enable support for the Intel TDX on your node.

Configure Firewall​

Since you will be hosting 3rd party applications on your server, we strongly recommend to configure your firewall to prevent any local area network connections from ROFL apps.

Using iptables, running something like this will prevent Oasis node and other processes owned by the oasis user accessing LAN on 192.168.0.255 except for the gateway:

iptables -A OUTPUT -d 192.168.0.1/32 -m owner --uid-owner $(id -u oasis) -j ACCEPT
iptables -A OUTPUT -d 192.168.0.0/16 -m owner --uid-owner $(id -u oasis) -j DROP

You can also permanently store the rules above. On Debian-based OSes you can do so by running:

sudo apt install iptables-persistent
sudo /etc/init.d/iptables-persistent save

Fund Your Node​

The node will also need to cover any transaction fees required to maintain registration of the ROFL node and the apps. First, determine the address of the account corresponding to your node:

oasis-node identity show-address -a unix:/node/data/internal.sock
oasis1qp6tl30ljsrrqnw2awxxu2mtxk0qxyy2nymtsy90

Then fund this account on Sapphire by transferring or depositing tokens with the oasis account command, for example:

oasis account transfer 10 oasis1qp6tl30ljsrrqnw2awxxu2mtxk0qxyy2nymtsy90

If you are just testing out your node on Sapphire Testnet, you can also request some TEST from the Testnet Faucet.

Configuration​

There are two ways you can host ROFL apps on your ROFL node. The preferred way is to join a network of ROFL providers called the ROFL marketplace and which is also integrated in the oasis rofl deploy command. Alternatively, you can copy ROFL bundle(s) directly to your node and configure each one of them in your node configuration file.

Hosting via ROFL marketplace​

To make your ROFL node accessible through ROFL marketplace, you will:

  1. Create a new ROFL provider entity.
  2. Configure one or more ROFL nodes to execute ROFL transactions corresponding to that provider and/or machine.

Both steps take place solely on-chain. There is no centralized mechanism or KYC process involved at any time.

Register your ROFL provider​

To register a new ROFL provider, using the Oasis CLI to run the following command:

oasis rofl provider init

Then edit your rofl-provider.yaml and add one or more hosting offers. Now obtain your Node ID and decide how much resources you are willing to lend out. In the example below, we'll offer small instances with 4 GiB of memory, 2 CPUs and 20 GB of storage. Hourly rent will cost 10 tokens and there can be at most 50 active instances at a time.

rofl-provider.yaml
network: testnet
paratime: sapphire
provider: test:erin
nodes:
  - 5MsgQwijUlpH9+0Hbyors5jwmx7tTmKMA4c9leV3prI=
scheduler_app: rofl1qrqw99h0f7az3hwt2cl7yeew3wtz0fxunu7luyfg
payment_address: test:erin
offers:
  - id: small
    resources:
      tee: tdx
      memory: 4096
      cpus: 2
      storage: 20000
    payment:
      native:
        terms:
          hourly: 10
    capacity: 50

To register a new provider using the configuration above, run:

oasis rofl provider create

The account signing the transaction is now registered as a ROFL provider on-chain. In our case, the built-in test:erin account which we used for signing has address oasis1qqcd0qyda6gtwdrfcqawv3s8cr2kupzw9v967au6.

info

Registering a new ROFL provider requires depositing 100 tokens which are returned to you, when you deregister it.

Configure your ROFL node for the marketplace​

  1. Download the latest release of the Scheduler ROFL app and save it to your ROFL node, for example /node/rofls/. This app will listen to ROFL hosting requests, configure any incoming ROFLs and spin them up. It will also listen to ROFL admin request for stopping or restarting the ROFL machines.

  2. Add the Scheduler ROFL app to your Oasis node's config.yml inside the runtime.paths and configure the scheduler specifics such as the provider address, acceptable offers and capacities on this node:

    config.yml
    runtime:
      sgx:
        loader: /node/bin/oasis-core-runtime-loader
      paths:
        - /node/rofls/rofl-scheduler.testnet.orc
      runtimes:
        # Sapphire Testnet RONL with SGX
        - id: "000000000000000000000000000000000000000000000000a6d1e3ebf60dff6c"
          config:
            allowed_queries:
              - all_expensive: true
          components:
           - id: rofl.rofl1qrqw99h0f7az3hwt2cl7yeew3wtz0fxunu7luyfg # ROFL scheduler app ID, should not change
             permissions:
               - bundle_add
               - bundle_remove
               - volume_add
               - volume_remove
             config:
               rofl_scheduler:
                 provider_address: oasis1qqcd0qyda6gtwdrfcqawv3s8cr2kupzw9v967au6 # Your provider address
                 offers:
                   - small # Your offer name(s)
                 capacity:
                   instances: 24
                   memory: 65536
                   cpus: 24
                   storage: 549755813888

  3. Restart your Oasis node. After a while, your ROFL node will be ready to accept ROFLs. ROFL app developers can now simply deploy their ROFL to your node by providing the --provider <your address> to the oasis rofl deploy command.

    oasis rofl deploy --provider oasis1qqcd0qyda6gtwdrfcqawv3s8cr2kupzw9v967au6
Multiple ROFL nodes

If you configured multiple ROFL nodes for a single provider, the machine instantiated to execute the ROFL app will be arbitrarily picked depending on which ROFL node register transaction appears first on chain.

​

Multiple ROFL replicas on a single node

The ROFL scheduler supports running multiple replicas of the same ROFL app on the same ROFL node by remapping the ROFL app ID to a unique value on each deployment. Look for the starting processor message in your logs to figure out the remapped value, for example:

{
  "app_id":"rofl1qrjtky678pd3uchsdlhqtjugnsvtck3wyg7w5324",
  "component":"rofl.4bd2d31255ae7e5cec31084cde02fb40640d4d678db111d1c6ba53478f5f2fc2",
  "msg":"starting processor",
  ...
}

Above, the original ROFL app ID rofl1qrjtky678pd3uchsdlhqtjugnsvtck3wyg7w5324 was remapped to 4bd2d31255ae7e5cec31084cde02fb40640d4d678db111d1c6ba53478f5f2fc2.

Hosting the ROFL App Bundle Directly​

To execute a ROFL app on your node, simply copy it over to your node, for example inside the /node/rofls folder. Then, put the location of the ORC bundle to the runtime.paths section of your configuration similarly to how other ParaTimes can be enabled on your node:

config.yml
runtime:
  # ... other options omitted ...
  paths:
    - /node/rofls/myapp.default.orc

Check that the path to your ROFL app bundle is correct. After starting your node, please make sure that the node is fully synchronized with Sapphire.

Exposing the ports​

To expose a specific TCP port of a ROFL app externally, add the following configuration to your Oasis node config:

config.yml
runtime:
  runtimes:
    - id: "000000000000000000000000000000000000000000000000a6d1e3ebf60dff6c"
      components:
        - id: rofl.rofl1qpkplp3uq5yage4kunt0ylmulett0arzwcdjvc8u # Your ROFL app ID
          networking:
            incoming:
              - ip: 192.168.0.10
                protocol: tcp
                src_port: 443
                dst_port: 443

In the example above, we exposed a TCP port 443 externally on the IP address 192.168.0.10 of our host.

Persistent storage​

The encrypted persistent storage of each ROFL app replica lives in /node/data/runtimes/volumes/{random hex value} folder.

It is generated when a ROFL app is executed for the first time and will remain intact even during ROFL upgrades and removal. Beside is also a descriptor.json that contains information which ROFL app does this volume belong to and other metadata.

Checking status​

You can check the logs of any ROFL app by grepping its app ID in your Oasis node log file. Since a Scheduler app to manage your ROFLs is also just a ROFL app, you can check if there are any issues reported by Scheduler by grepping for the its app ID:

grep rofl.rofl1qrqw99h0f7az3hwt2cl7yeew3wtz0fxunu7luyfg /node/data/node.log

To extract only the relevant msg field you may do:

grep rofl.rofl1qrqw99h0f7az3hwt2cl7yeew3wtz0fxunu7luyfg /node/data/node.log | jq -r '.msg'

When exploring the logs, keep in mind that the ROFL app IDs of the Scheduler-managed apps will be remapped.

See also​

📄️ Deployment

ROFLs can be deployed to any ParaTime that has the ROFL module installed. In our