gpustat-web
A web interface of gpustat
---
aggregate gpustat
across multiple nodes.
NOTE: This project is in alpha stage. Errors and exceptions are not well handled, and it might use much network resources. Please use at your own risk!
Installation
pip install gpustat-web
Python 3.6+ is required.
Usage
Launch the application as follows. A SSH connection will be established to each of the specified hosts.
Make sure ssh <host>
works under a proper authentication scheme such as SSH authentication.
gpustat-web --port 48109 HOST1 [... HOSTN]
You might get "Host key is not trusted for <host>
" errors. You'll have to accept and trust SSH keys of the host for the first time (it's stored in ~/.ssh/known_hosts
);
try ssh <host>
in the command line, or ssh -oStrictHostKeyChecking=accept-new <host>
to automatically accept the host key. You can also use an option gpustat-web --no-verify-host
to bypass SSH Host key validation (although not recommended).
Note that asyncssh does NOT obey the ~/.ssh/config
file
(e.g. alias, username, keyfile), so any config in ~/.ssh/config
might not be picked up.
Endpoints
https://HOST:PORT/
: A webpage that updates automatically through websocket.https://HOST:PORT/gpustat.html
: Result as a static HTML page.https://HOST:PORT/gpustat.txt
: Result as a static plain text.https://HOST:PORT/gpustat.ansi
: Result as a static text with ANSI color codes. Trycurl https://.../gpustat.ansi
Query strings:
?nodes=gpu001,gpu002
: Select a subset of nodes to query and display
Running as a HTTP (SSL/TLS) server
By default the web server will run as a HTTP server.
If you want to run a secure SSL/TLS server over the HTTPS protocol, use --ssl-certfile
and --ssl-keyfile
option.
You can use letsencrypt (certbot
) to create a pair of SSL certificate and keyfile.
Troubleshoothing: Verify SSL/TLS handshaking (if TLS connections cannot be established)
openssl s_client -showcerts -connect YOUR_HOST.com:PORT < /dev/null
More Examples
To see CPU usage as well:
python -m gpustat_web --exec 'gpustat --color --gpuname-width 25 && echo -en "CPU : \033[0;31m" && cpu-usage | ascii-bar 27'
License
MIT License
Copyright (c) 2018-2023 Jongwook Choi