Fly.io (experimental)

caution

This deployment guide is experimental and may not reliably work each time or for every user. We came across issues when deploying to Fly.io, which may render your application not behaving as expected. You can learn more about the problems we came across in the deploying section

Fly.io is the go-to platform for deploying Phoenix apps. This guide will help you deploy Jellyfish server on Fly.io.

First, we recommend you read the Fly.io speedrun for deploying an app.

Fly.io uses fly.toml files for configuring an app. You can start from our fly.toml sample file:

fly.toml

kill_signal = "SIGTERM"
kill_timeout = 5
processes = []

[env]
  JF_PORT = "4000"
  JF_WEBRTC_USED = "true"
  JF_WEBRTC_TURN_IP = "<ip obtained with fly ips allocate-v4>"
  JF_WEBRTC_TURN_PORT_RANGE = "50000-59999"
  JF_WEBRTC_TURN_LISTEN_IP = "fly-global-services"

[experimental]
  auto_rollback = true

[[services]]
  http_checks = []
  internal_port = 4000
  protocol = "tcp"
  script_checks = []
  [services.concurrency]
    hard_limit = 1000
    soft_limit = 1000
    type = "connections"

  [[services.ports]]
    force_https = true
    handlers = ["http"]
    port = 80

  [[services.ports]]
    handlers = ["tls", "http"]
    port = 443

  [[services.tcp_checks]]
    grace_period = "1s"
    interval = "15s"
    restart_limit = 0
    timeout = "2s"

[[services]]
  internal_port = 50000
  protocol = "udp"

  [[services.ports]]
    port = 50000

Creating new app

Launch fly launch. When prompted, copy the configuration from the existing fly.toml file.

Select appropriate app name and region, don't create any databases. Don't deploy the app just yet.

Running fly launch may modify the Dockerfile. Make sure to remove the lines appended by flyctl.

Configuring the App

Before deploying your app, make sure it is correctly configured.

Dedicated IPv4

In order to transmit multimedia via UDP, the app has to be publicly available with an IPv4 address. By default Fly.io uses shared IP. To enable dedicated IPv4 address run

fly ips allocate-v4

Note, that dedicated IP address is a paid feature on Fly.io.

You can learn more about dedicated IPv4 address here.

fly.toml

Make sure you don't have the following lines in the fly.toml, which are automatically generated by fly launch.

You don't need to run migrations, since you don't have a database.

[deploy]
  release_command = "/app/bin/migrate"

Jellyfish uses JF_HOST variable instead of the default PHX_HOST.

JF_HOST = "<YOUR APP HOSTNAME>"

Also, make sure you have set the correct port. The environment variable JF_PORT has to match the TCP internal_port defined under services section. The default for Jellyfish is 5002 in development and 8080 in production (when using Docker or mix release).

To be able to receive and send UDP traffic, Jellyfish has to open its UDP ports on a special fly-global-services address, not 0.0.0.0.

This must be set using the JF_WEBRTC_TURN_LISTEN_IP enviroment variable. You also need to specify the Jellyfish IP address for UDP, it is the IP address which you generated in the previous step.

JF_WEBRTC_TURN_LISTEN_IP = "fly-global-services"
JF_WEBRTC_TURN_IP="<YOUR APP IP ADDRESS>"

You can also read tutorial for running Fly.io apps which use UDP.

Fly.io secrets

There are environment variables, which you may not want to keep in the fly.toml config. Fly.io provides a way to store such values securely.

For Jellyfish you need to configure just one secret - JF_SERVER_API_TOKEN.

flyctl secrets set JF_SERVER_API_TOKEN=development

Deploying

With everything configured you can deploy the app

fly deploy

Note that it may take a moment for the UDP traffic to be forwarded to the application. This means for example, that WebRTC may not be working yet.

This is the tricky part of the deployment, which we weren't able to entirely figure out. Sometimes the UDP just works, others it takes ages for it to start flowing. This topic describes the exact issue we had with Fly.io.

In general, it is unclear when the UDP traffic starts to be forwarded, sometimes it happens relatively quickly after deployment, sometimes you need to wait for some time.

Also, the Fly.io documentation isn't clear in regard to opening UDP sockets. Some information was gathered experimentally, or based on Fly.io community questions, such as this one.