Launching Graphistry in AWS Marketplace? Get started with the walkthrough tutorial and videos!
- Marketplace home: Click Continue to subscribe, Continue to Configuration
- Do not Configure contract as that sets an annual prepay
- Page Configure this software:
- Region: Select an AWS region with GPUs such as US East or Oregon (see AWS availability of p3.* instances)
- You may need to request capacity for your account to run GPUs
- Do not Configure contract as that sets an annual prepay
- Click Continue to launch
- Page Launch this software:
- EC2 Instance Type: Pick g4dn.2xlarge+ (8 vCPU + 1 T4 GPU) / p3.2xlarge+ (8 vCPU + 1 V100 GPU) or larger as Graphistry/RAPIDS require a Pascal or later GPU. For single-user testing, you can pick g4dn.xlarge (4 vCPU + 1 T4 GPU).
- VPC Settings, Subnet Settings: Pick something that your browser/client can access (http/https/ssh) and can speak to your DB
- Key Pair Settings: Reuse or create a Key Pair so you can SSH to the commandline for administration
- Click Launch
- Go to the public IP in a browser and refresh until the login screen loads, login with
admin/i-YourAWSInstanceID, and start graphing!
- Security: Enable auto-TLS and restrict network access
- Create users
- Generate API keys for individuals without accounts
- Turn server on-and-off via AWS Console via stop and start
- Advanced configuration
- Update, backup, and migrate
- To simplify administration and limit downtime, we recommend creating a new Marketplace instance, copying data snapshots to it and loading it in, and switching DNS to the new instance only when tested
The Graphistry marketplace instance is designed for secure web-based use and administration. However, command-line administration can be helpful. This document shares common marketplace tasks. See the main docs for general CLI use.
Contents:
- Recommended configuration
- Solve GPU availability errors
- Command-line Login
- Docker
- Install Python packages
- Install native packages
- Marketplace FAQ
- Use a regular AWS account as it is safer than AWS Root account
- Associate your AWS instance with an Elastic IP or a domain
- Setup TLS, potentially through an AWS Load Balancer (ALB)
- In restricted environments, constrain networking to a safelist, e.g., VPN, and optional, change logging drivers to stop Graphistry from recieving maintenace logs. See recommended port restrictions
- You can limit instance traffic to VPC-internal, such as not associating a public IP: SSH via a bastion server, and route all public HTTP/HTTPS through an AWS Load Balancer (ALB), including aan AWS Web Application Firewall (WAF)
- If expecting many uploads, attach a managed disk. Contact Graphistry staff for automation assistance.
Upon trying to launch, Amazon may fail with an error about no available GPUs for two reasons:
-
Lack of GPU availability in the current region. In this case, try another valid GPU type, or launching in another region. For example, Virginia => Oregon. Keeping the GPU close to your users is a good idea to minimize latency.
-
Insufficient account quota. In this case, the error should also contain a link to increase your quota. Request
p3.2(and above), and 1-2 for a primary region and 1-2 for a secondary region.
Log in using the key configured at AWS instance start and your instance's public IP/domain:
ssh -i my_key.pem ubuntu@MY_PUBLIC_IP_HERE
Many ssh clients may require you to first run chmod 400 my_key.pem or chmod 644 my_key.pem before running the above.
Graphistry leverages docker-compose and the AWS Marketplace AMI preconfigures the nvidia runtime for docker.
cd ~/graphistry
sudo docker-compose ps
=>
Name Command State Ports
------------------------------------------------------------------------------------------------------------------------------------
compose_caddy_1 /bin/parent caddy --conf / ... Up 2015/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:80->80/tcp
compose_forge-etl-python_1 /tini -- /entrypoints/etl- ... Up (unhealthy) 8080/tcp
compose_forge-etl_1 /tini -- /entrypoints/pm2. ... Up (healthy) 8080/tcp
compose_nexus_1 /entrypoint /bin/bash -c b ... Up 8000/tcp
compose_nginx_1 nginx -g daemon off; Up 80/tcp, 8080/tcp
compose_notebook_1 /tini -g -- /bin/bash -c s ... Up 8080/tcp
compose_pivot_1 /tini -- /entrypoints/stre ... Up (healthy) 8080/tcp
compose_postgres_1 docker-entrypoint.sh postgres Up 5432/tcp, 8080/tcp
compose_redis_1 docker-entrypoint.sh redis ... Up 6379/tcp, 8080/tcp
compose_streamgl-gpu_1 /tini -- /entrypoints/fast ... Up (healthy) 8080/tcp
compose_streamgl-sessions_1 /tini -- /entrypoints/fast ... Up (healthy) 8080/tcp
compose_streamgl-vgraph-etl_1 /tini -- /entrypoints/fast ... Up (healthy) 8080/tcp
compose_streamgl-viz_1 /tini -- /entrypoints/stre ... Up 8080/tcp
Note: Precise set of containers changes across versions
If you see wheel errors, you may need to run pip install wheel and restart your Jupyter kernel.
By default, Jupyter users do not have sudo, restricting them to user-level installation like pip. For system-level actions, such as for installing golang and other tools, you can create interactive root user sessions by logging into the Jupyter Docker container:
Admin:
Note that sudo is unnecessary within the container:
ubuntu@ip-172-31-0-38:~/graphistry$ docker exec -it -u root graphistry_notebook_1 bash
root@d4afa8b7ced5:/home/graphistry# apt update
root@d4afa8b7ced5:/home/graphistry# apt install golang
root@d4afa8b7ced5:/home/graphistry# source activate rapids && conda install pyarrow
User:
ubuntu@ip-172-31-0-38:~/graphistry$ docker exec -it graphistry_notebook_1 bash
graphistry@d4afa8b7ced5:~$ go version
=>
go version go1.10.4 linux/amd64
Wait a few minutes for the system to finish starting. If the problem persists for more than 5-10min, log in, run docker ps, and for each failing service, restart it. If problems persist further, please report the results of docker logs <service> to the Graphistry support team and we will help out.
See the reset command in the main README. Requires SSH'ing in, and will delete all users, but not data.
See section login
See general installation for further information.