server.py - Gate One's Core Script

Gate One

Gate One is a web-based terminal emulator written in Python using the Tornado web framework. This module runs the primary daemon process and acts as a central controller for all running terminals and terminal programs. It supports numerous configuration options and can also be called with the --kill switch to kill all running terminal programs (if using dtach--otherwise they die on their own when gateone.py is stopped).

Dependencies

Gate One requires Python 2.6+ but runs best with Python 2.7+. It also depends on the following 3rd party Python modules:

  • Tornado 3.1+ - A non-blocking web server framework that powers FriendFeed.

The following modules are optional and can provide Gate One with additional functionality:

  • pyOpenSSL 0.10+ - An OpenSSL module/wrapper for Python. Only used to generate self-signed SSL keys and certificates. If pyOpenSSL isn't available Gate One will fall back to using the 'openssl' command to generate self-signed certificates.
  • kerberos 1.0+ - A high-level Kerberos interface for Python. Only necessary if you plan to use the Kerberos authentication module.
  • python-pam 0.4.2+ - A Python module for interacting with PAM (the Pluggable Authentication Module present on nearly every Unix). Only necessary if you plan to use PAM authentication.
  • PIL (Python Imaging Library) 1.1.7+ - A Python module for manipulating images. Alternative: Pillow 2.0+ - A "friendly fork" of PIL that works with Python 3.
  • mutagen 1.21+ - A Python module to handle audio metadata. Makes it so that if you cat music_file.ogg in a terminal you'll get useful track/tag information.

With the exception of python-pam, all required and optional modules can usually be installed via one of these commands:

user@modern-host:~ $ sudo pip install --upgrade tornado pyopenssl kerberos pillow mutagen

...or:

user@legacy-host:~ $ sudo easy_install tornado pyopenssl kerberos pillow mutagen

Note

The use of pip is recommended. See http://www.pip-installer.org/en/latest/installing.html if you don't have it.

The python-pam module is available in most Linux distribution repositories. Simply executing one of the following should take care of it:

user@debian-or-ubuntu-host:~ $ sudo apt-get install python-pam
user@redhat-host:~ $ sudo yum install python-pam
user@gentoo-host:~ $ sudo emerge python-pam
user@suse-host:~ $ sudo yast -i python-pam

Settings

Most of Gate One's options can be controlled by command line switches or via .conf files in the settings directory. If you haven't configured Gate One before a number of .conf files will be automatically generated using defaults and/or the command line switches provided the first time you run gateone.py.

Settings in the various settings/*.conf files are JSON-formatted:

Note

Technically, JSON doesn't allow comments but Gate One's .conf files do.

{ // You can use single-line comments like this
/*
Or multi-line comments like this.
*/
    "*": { // The * here designates this as "default" values
        "gateone": { // Settings in this dict are specific to the "gateone" app
            "address": "10.0.0.100", // A string value
            "log_file_num_backups": 10, // An integer value
            // Here's an example list:
            "origins": ["localhost", "127.0.0.1", "10.0.0.100"],
            "https_redirect": false, // Booleans are all lower case
            "log_to_stderr": null // Same as `None` in Python (also lower case)
            // NOTE: No comma after last item
        },
        "terminal": { // Example of a different application's settings
            "default_command": "SSH", // <-- don't leave traling commas like this!
            ... // So on and so forth
        }
    }
}

Note

You must use double quotes ("") to define strings. Single quotes can be used inside double quotes, however. "example": "of 'single' quotes" To escape a double-quote inside a double quote use three slashes: "\\\"foo\\\""

You can have as many .conf files in your settings directory as you like. When Gate One runs it reads all files in alphanumeric order and combines all settings into a single Python dictionary. Files loaded last will override settings from earlier files. Example:

20example.conf

{
    "*": {
        "terminal": {
            "session_logging": true,
            "default_command": "SSH"
        }
    }
}

99override.conf

{
    "*": {
        "terminal": {
            "default_command": "my_override"
        }
    }
}

If Gate One loaded the above example .conf files the resulting dict would be:

Merged settings

{
    "*": {
        "terminal": {
            "session_logging": true,
            "default_command": "my_override"
        }
    }
}

Note

These settings are loaded using the RUDict (Recusive Update Dict) class.

There are a few important differences between the configuration file and command line switches in regards to boolean values (True/False). A switch such as --debug is equivalent to "debug" = true in 10server.conf:

"debug" = true // Booleans are all lower case (not in quotes)

Tip

Use --setting=True, --setting=False, or --setting=None to avoid confusion.

Note

The following values in 10server.conf are case sensitive: true, false and null (and should not be placed in quotes).

Gate One's configuration files also provide access control mechanisms. These are controlled by setting the scope of the configuration block. In this example all users that match the given IP address will be denied access to Gate One:

{
    "user.ip_address=(127.0.0.1|10.1.1.100)": { // Replaces "*"
        "gateone": { // These settings apply to all of Gate One
            "blacklist": true,
        }
    }
}

You can define scopes however you like using user's attributes. For example:

{
    "user.email=.*@company.com": {
        "terminal": {
            "commands": {"extra": "/some/extra/command.sh"}
        }
    }
}

The above example would make it so that all users with an email address ending in '@company.com' will get access to the "extra" command when using the "terminal" application.

Note

Different authentication types provide different user attributes. For example, if you have "auth" set to "google" authenticated users will have a 'gender' attribute. So for example--if you wanted to be evil--you could provide different settings for men and women (e.g. "user.gender=male").

Tip

When using API authentication you can pass whatever extra user attributes you want via the 'auth' object. These attributes will be automatically assigned to the user and can be used with the policy mechanism to control access and settings.

Running gateone.py with the --help switch will print the usage information as well as descriptions of what each configurable option does:

$ gateone --help
[E 160618 14:04:28 __init__:18] In order to use the X11 application you must install the xpyb module.  This can usually be handled via:  sudo pip install xpyb ||  sudo apt-get install python-xpyb
Usage: /usr/local/bin/gateone [OPTIONS]

Options:


/usr/local/lib/python3.5/dist-packages/gateone-1.2.0-py3.5.egg/gateone/core/configuration.py options:

  --help                           Show this help information

/usr/local/lib/python3.5/dist-packages/tornado-4.3-py3.5-linux-x86_64.egg/tornado/log.py options:

  --log_file_max_size              max size of log files before rollover (default 100000000)
  --log_file_num_backups           number of log files to keep (default 10)
  --log_file_prefix=PATH           Path prefix for log files. Note that if you are running multiple tornado processes, log_file_prefix must be different for each of them (e.g. include the port number)
  --log_rotate_interval            The interval value of timed rotating (default 1)
  --log_rotate_mode                The mode of rotating files(time or size) (default size)
  --log_rotate_when                specify the type of TimedRotatingFileHandler interval other options:('S', 'M', 'H', 'D', 'W0'-'W6') (default midnight)
  --log_to_stderr                  Send log output to stderr (colorized if possible). By default use stderr if --log_file_prefix is not set and no other logging is configured.
  --logging=debug|info|warning|error|none 
                                   Set the Python log level. If 'none', tornado won't touch the logging configuration. (default info)

gateone options:

  --address                        Run on the given address.  Default is all addresses (IPv6 included).  Multiple address can be specified using a semicolon as a separator (e.g. '127.0.0.1;::1;10.1.1.100').
  --api_keys                       The 'key:secret,...' API key pairs you wish to use (only applies if using API authentication)
  --api_timestamp_window           How long before an API authentication object becomes invalid. (default 30s)
  --auth                           Authentication method to use.  Valid options are: none, api, cas, google, ssl, pam (default none)
  --ca_certs                       Path to a file containing any number of concatenated CA certificates in PEM format.  They will be used to authenticate clients if the 'ssl_auth' option is set to 'optional' or 'required'.
  --cache_dir                      Path where Gate One should store temporary global files (e.g. rendered templates, CSS, JS, etc). (default /home/riskable/.gateone/cache)
  --certificate                    Path to the SSL certificate.  Will be auto-generated if not found. (default /etc/gateone/ssl/certificate.pem)
  --combine_css                    Combines all of Gate One's CSS Template files into one big file and saves it to the given path (e.g. ./gateone.py --combine_css=/tmp/gateone.css).
  --combine_css_container          Use this setting in conjunction with --combine_css if the <div> where Gate One lives is named something other than #gateone (default gateone)
  --combine_js                     Combines all of Gate One's JavaScript files into one big file and saves it to the given path (e.g. ./gateone.py --combine_js=/tmp/gateone.js)
  --command                        DEPRECATED: Use the 'commands' option in the terminal settings.
  --config                         DEPRECATED.  Use --settings_dir. (default /opt/gateone/server.conf)
  --configure                      Only configure Gate One (create SSL certs, conf.d, etc).  Do not start any Gate One processes. (default False)
  --cookie_secret                  Use the given 45-character string for cookie encryption.
  --debug                          Enable debugging features such as auto-restarting when files are modified. (default False)
  --disable_ssl                    If enabled Gate One will run without SSL (generally not a good idea). (default False)
  --embedded                       When embedding Gate One this option is available to plugins, applications, and templates so they know they're running in embedded mode and can change behavior (if necessary). (default False)
  --enable_unix_socket             Enable Unix socket support. (default False)
  --gid                            Drop privileges and run Gate One as this group/gid. (default 1000)
  --https_redirect                 If enabled a separate listener will be started on port 80 that redirects users to the configured port using HTTPS. (default False)
  --js_init                        A JavaScript object (string) that will be used when running GateOne.init() inside index.html.  Example: --js_init="{theme: 'white'}" would result in GateOne.init({theme: 'white'})
  --keyfile                        Path to the SSL keyfile.  Will be auto-generated if none is provided. (default /etc/gateone/ssl/keyfile.pem)
  --locale                         The locale (e.g. pt_PT) Gate One should use for translations.  If not provided, will default to $LANG (which is 'en_US' in your current shell). (default en_US)
  --multiprocessing_workers        The number of processes to spawn use when using multiprocessing. Default is: <number of cores> + 1.  Set to 0 to disable multiprocessing.
  --new_api_key                    Generate a new API key that an external application can use to embed Gate One. (default False)
  --origins                        A semicolon-separated list of origins you wish to allow access to your Gate One server over the WebSocket.  This value may contain hostnames/FQDNs (e.g. foo;foo.bar;) and IP addresses.  This
                                   value must contain all the hostnames/IPs that users will use to connect to Gate One.  Alternatively, '*' may be  specified to allow access from anywhere.  NOTE: Using a '*' is only a good idea
                                   if you've configured Gate One to use API authentication. (default localhost;127.0.0.1;somehost.example.com;somehost;127.0.1.1)
  --pam_realm                      Basic auth REALM to display when authenticating clients.  Default: hostname.  Only relevant if PAM authentication is enabled. (default somehost)
  --pam_service                    PAM service to use.  Defaults to 'login'. Only relevant if PAM authentication is enabled. (default login)
  --pid_file                       Define the path to the pid file. (default /home/riskable/.gateone/gateone.pid)
  --port                           Run on the given port. (default 10443)
  --session_dir                    Path to the location where session information will be stored. (default /home/riskable/.gateone/sessions)
  --session_timeout                Amount of time that a session is allowed to idle before it is killed.  Accepts <num>X where X could be one of s, m, h, or d for seconds, minutes, hours, and days.  Set to '0' to disable the
                                   ability to resume sessions. (default 5d)
  --settings_dir                   Path to the settings directory. (default /home/riskable/.gateone/conf.d)
  --ssl_auth                       Enable the use of client SSL (X.509) certificates as a secondary authentication factor (the configured 'auth' type will come after SSL auth).  May be one of 'none', 'optional', or 'required'.
                                   NOTE: Only works if the 'ca_certs' option is configured. (default none)
  --sso_realm                      Kerberos REALM (aka DOMAIN) to use when authenticating clients.  Only relevant if Kerberos authentication is enabled.
  --sso_service                    Kerberos service (aka application) to use.  Only relevant if Kerberos authentication is enabled. (default HTTP)
  --syslog_facility                Syslog facility to use when logging to syslog (if syslog_session_logging is enabled).  Must be one of: auth, cron, daemon, kern, local0, local1, local2, local3, local4, local5, local6, local7,
                                   lpr, mail, news, syslog, user, uucp. (default daemon)
  --uid                            Drop privileges and run Gate One as this user/uid. (default 1000)
  --unix_socket_mode               Unix socket mode (if --enable_unix_socket=True). (default 0600)
  --unix_socket_path               Path to the Unix socket (if --enable_unix_socket=True). (default /tmp/gateone.sock)
  --url_prefix                     An optional prefix to place before all Gate One URLs. e.g. '/gateone/'.  Use this if Gate One will be running behind a reverse proxy where you want it to be located at some sub-URL path.
                                   (default /)
  --user_dir                       Path to the location where user files will be stored. (default /home/riskable/.gateone)
  --user_logs_max_age              Maximum length of time to keep any given user log before it is automatically removed. (default 30d)
  --version                        Display version information.

terminal options:

  --dtach                          Wrap terminals with dtach. Allows sessions to be resumed even if Gate One is stopped and started (which is a sweet feature). (default True)
  --kill                           Kill any running Gate One terminal processes including dtach'd processes. (default False)
  --session_logging                If enabled, logs of user sessions will be saved in <user_dir>/<user>/logs.  Default: Enabled (default True)
  --syslog_session_logging         If enabled, logs of user sessions will be written to syslog. (default False)

Commands:

  Usage: /usr/local/bin/gateone <command> [OPTIONS]

  GateOne supports many different CLI 'commands' which can be used to invoke special functionality provided by plugins and applications (and application's plugins).  Each command can have it's own options and most will have a
  --help function of their own.

Commands provided by 'gateone':

  broadcast                        Broadcast messages to users connected to Gate One.
  install_license                  Install a commercial license for Gate One.
  validate_authobj                 Test API authentication by validating a JSON auth object.

Commands provided by 'gateone.applications.terminal':

  termlog                          View terminal session logs.

Example command usage:

  /usr/local/bin/gateone termlog --help

File Paths

Gate One stores its files, temporary session information, and persistent user data in the following locations (Note: Many of these are configurable):

File/Directory Description
authpam.py Contains the PAM authentication Mixin used by auth.py.
auth.py Authentication classes.
babel_gateone.cfg Pybabel configuration for generating localized translations of Gate One's strings.
certificate.pem The default certificate Gate One will use in SSL communications.
docs/ Gate One's documentation.
gateone.py Gate One's primary executable/script. Also, the file containing this documentation.
gopam.py PAM (Authentication) Python module (used by authpam.py).
i18n/ Gate One's internationalization (i18n) support and locale/translation files.
keyfile.pem The default private key used with SSL communications.
logviewer.py A utility to view Gate One session logs.
plugins/ (Global) Plugins go here in the form of ./plugins/<plugin name>/<plugin files|directories>
settings/ All Gate One settings files live here (.conf files).
sso.py A Kerberos Single Sign-on module for Tornado (used by auth.py)
static/ Non-dynamic files that get served to clients (e.g. gateone.js, gateone.css, etc).
templates/ Tornado template files such as index.html.
tests/ Various scripts and tools to test Gate One's functionality.
utils.py Various supporting functions.
users/ Persistent user data in the form of ./users/<username>/<user-specific files>
users/<user>/logs This is where session logs get stored if session_logging is set.
/tmp/gateone Temporary session data in the form of /tmp/gateone/<session ID>/<files>
/tmp/gateone_cache Used to store cached files such as minified JavaScript and CSS.

Running

Executing Gate One is as simple as:

root@host:~ $ gateone

Note

By default Gate One will run on port 443 which requires root on most systems. Use --port=(something higher than 1024) for non-root users.

Applications and Plugins

Gate One supports both applications and plugins. The difference is mostly architectural. Applications go in the gateone/applications directory while (global) plugins reside in gateone/plugins. The scope of application code applies only to the application wheras global Gate One plugin code will apply to Gate One itself and all applications.

Note

Applications may have plugins of their own (e.g. terminal/plugins).

Gate One applications and plugins can be written using any combination of the following:

  • Python
  • JavaScript
  • CSS

Applications and Python plugins can integrate with Gate One in a number ways:

  • Adding or overriding request handlers to provide custom URLs (with a given regex).
  • Adding or overriding methods in GOApplication to handle asynchronous WebSocket "actions".
  • Adding or overriding events via the on(), off(), once(), and trigger() functions.
  • Delivering custom content to clients (e.g. JavaScript and CSS templates).

JavaScript and CSS plugins will be delivered over the WebSocket and cached at the client by way of a novel synchronization mechanism. Once JavaScript and CSS assets have been delivered they will only ever have to be re-delivered to clients if they have been modified (on the server). This mechanism is extremely bandwidth efficient and should allow clients to load Gate One content much more quickly than traditional HTTP GET requests.

Tip

If you install the cssmin and/or slimit Python modules all JavaScript and CSS assets will be automatically minified before being delivered to clients.

Class Docstrings

gateone.core.server._(string)[source]

Wraps server_locale.translate so we don't get errors if loading a locale fails (or we output a message before it is initialized).

gateone.core.server.cleanup_user_logs()[source]

Cleans up all user logs (everything in the user's 'logs' directory and subdirectories that ends in 'log') older than the user_logs_max_age setting. The log directory is assumed to be:

user_dir/<user>/logs

...where user_dir is whatever Gate One happens to have configured for that particular setting.

gateone.core.server.cleanup_old_sessions()[source]

Cleans up old session directories inside the session_dir. Any directories found that are older than the auth_timeout (global gateone setting) will be removed. The modification time is what will be checked.

gateone.core.server.clean_up()[source]

Regularly called via the CLEANER PeriodicCallback, calls cleanup_user_logs and cleanup_old_sessions.

Note

How often this function gets called can be controlled by adding a cleanup_interval setting to 10server.conf ('gateone' section).

gateone.core.server.policy_send_user_message(cls, policy)[source]

Called by gateone_policies(), returns True if the user is authorized to send messages to other users and if applicable, all users (broadcasts).

gateone.core.server.policy_broadcast(cls, policy)[source]

Called by gateone_policies(), returns True if the user is authorized to broadcast messages using the ApplicationWebSocket.broadcast() method. It makes this determination by checking the ['gateone']['send_broadcasts'] policy.

gateone.core.server.policy_list_users(cls, policy)[source]

Called by gateone_policies(), returns True if the user is authorized to retrieve a list of the users currently connected to the Gate One server via the ApplicationWebSocket.list_server_users() method. It makes this determination by checking the ['gateone']['list_users'] policy.

gateone.core.server.gateone_policies(cls)[source]

This function gets registered under 'gateone' in the ApplicationWebSocket.security dict and is called by the require() decorator by way of the policies sub-function. It returns True or False depending on what is defined in the settings dir and what function is being called.

This function will keep track of and place limits on the following:

  • Who can send messages to other users (including broadcasts).
  • Who can retrieve a list of connected users.
gateone.core.server.kill_all_sessions(timeout=False)[source]

Calls all 'kill_session_callbacks' attached to all SESSIONS.

If timeout is True, emulate a session timeout event in order to really kill any user sessions (to ensure things like dtach processes get killed too).

gateone.core.server.timeout_sessions()[source]

Loops over the SESSIONS dict killing any sessions that haven't been used for the length of time specified in TIMEOUT (global). The value of TIMEOUT can be set in 10server.conf or specified on the command line via the session_timeout value.

Applications and plugins can register functions to be called when a session times out by attaching them to the user's session inside the SESSIONS dict under 'timeout_callbacks'. The best place to do this is inside of the application's authenticate() function or by attaching them to the go:authenticate event. Examples:

# Imagine this is inside an application's authenticate() method:
sess = SESSIONS[self.ws.session]
# Pretend timeout_session() is a function we wrote to kill stuff
if timeout_session not in sess["timeout_session"]:
    sess["timeout_session"].append(timeout_session)

Note

This function is meant to be called via Tornado's PeriodicCallback().

gateone.core.server.broadcast_message(args=['/usr/bin/sphinx-build', '-b', 'html', '-v', '-T', '-d', 'build/doctrees', 'source', 'build/html'], message='')[source]

Broadcasts a given message to all users in Gate One. If no message is given sys.argv will be parsed and everything after the word 'broadcast' will be broadcast.

class gateone.core.server.StaticHandler(application, request, **kwargs)[source]

An override of tornado.web.StaticFileHandler to ensure that the Access-Control-Allow-Origin header gets set correctly. This is necessary in order to support embedding Gate One into other web-based applications.

Note

Gate One performs its own origin checking so header-based access controls at the client are unnecessary.

initialize(path, default_filename=None, use_pkg=None)[source]

Called automatically by the Tornado framework when the StaticHandler class is instantiated; handles the usual arguments with the addition of use_pkg which indicates that the static file should attempt to be retrieved from that package via the pkg_resources module instead of directly via the filesystem.

set_extra_headers(path)[source]

Adds the Access-Control-Allow-Origin header to allow cross-origin access to static content for applications embedding Gate One. Specifically, this is necessary in order to support loading fonts from different origins.

Also sets the 'X-UA-Compatible' header to 'IE=edge' to enforce IE 10+ into standards mode when content is loaded from intranet sites.

options(path=None)[source]

Replies to OPTIONS requests with the usual stuff (200 status, Allow header, etc). Since this is just the static file handler we don't include any extra information.

validate_absolute_path(root, absolute_path)[source]

An override of tornado.web.StaticFileHandler.validate_absolute_path();

Validate and returns the given absolute_path using pkg_resources if self.use_pkg is set otherwise performs a normal filesystem validation.

class gateone.core.server.BaseHandler(application, request, **kwargs)[source]

A base handler that all Gate One RequestHandlers will inherit methods from.

Provides the get_current_user() method, sets default headers, and provides a default options() method that can be used for monitoring purposes and also for enumerating useful information about this Gate One server (see below for more info).

set_default_headers()[source]

An override of tornado.web.RequestHandler.set_default_headers() (which is how Tornado wants you to set default headers) that adds/overrides the following headers:

Server:'GateOne'
X-UA-Compatible:
 'IE=edge' (forces IE 10+ into Standards mode)
get_current_user()[source]

Tornado standard method--implemented our way.

options(path=None)[source]

Replies to OPTIONS requests with the usual stuff (200 status, Allow header, etc) but also includes some useful information in the response body that lists which authentication API features we support in addition to which applications are installed. The response body is conveniently JSON-encoded:

user@modern-host:~ $ curl -k             -X OPTIONS https://gateone.company.com/ | python -mjson.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   158  100   158    0     0   6793      0 --:--:-- --:--:-- --:--:--  7181
{
    "applications": [
        "File Transfer",
        "Terminal",
        "X11"
    ],
    "auth_api": {
        "hmacs": [
            "HMAC-SHA1",
            "HMAC-SHA256",
            "HMAC-SHA384",
            "HMAC-SHA512"
        ],
        "versions": [
            "1.0"
        ]
    }
}

Note

The 'Server' header does not supply the version information. This is intentional as it amounts to an unnecessary information disclosure. We don't need to make an attacker's job any easier.

class gateone.core.server.HTTPSRedirectHandler(application, request, **kwargs)[source]

A handler to redirect clients from HTTP to HTTPS. Only used if https_redirect is True in Gate One's settings.

get()[source]

Just redirects the client from HTTP to HTTPS

class gateone.core.server.DownloadHandler(application, request, **kwargs)[source]

A tornado.web.RequestHandler to serve up files that wind up in a given user's session_dir in the 'downloads' directory. Generally speaking these files are generated by the terminal emulator (e.g. cat somefile.pdf) but it can be used by applications and plugins as a way to serve up all sorts of (temporary/transient) files to users.

class gateone.core.server.MainHandler(application, request, **kwargs)[source]

Renders index.html which loads Gate One.

Will include the minified version of gateone.js if available as gateone.min.js.

class gateone.core.server.GOApplication(ws)[source]

The base from which all Gate One Applications will inherit. Applications are expected to be written like so:

class SomeApplication(GOApplication):
    def initialize(self):
        "Called when the Application is instantiated."
        initialize_stuff()
        # Here's some good things to do in an initialize() function...
        # Register a policy-checking function:
        self.ws.security.update({'some_app': policy_checking_func})
        # Register some WebSocket actions (note the app:action naming convention)
        self.ws.actions.update({
            'some_app:do_stuff': self.do_stuff,
            'some_app:do_other_stuff': self.do_other_stuff
        })
    def open(self):
        "Called when the connection is established."
        # Setup whatever is necessary for session tracking and whatnot.
    def authenticate(self):
        "Called when the user *successfully* authenticates."
        # Here's the best place to instantiate things, send the user
        # JavaScript/CSS files, and similar post-authentication details.
    def on_close(self):
        "Called when the connection is closed."
        # This is a good place to halt any background/periodic operations.

GOApplications will be automatically imported into Gate One and registered appropriately as long as they follow the following conventions:

  • The application and its module(s) should live inside its own directory inside the 'applications' directory. For example, /opt/gateone/applications/some_app/some_app.py
  • Subclasses of GOApplication must be added to an apps global (list) inside of the application's module(s) like so: apps = [SomeApplication] (usually a good idea to put that at the very bottom of the module).

Note

All .py modules inside of the application's main directory will be imported even if they do not contain or register a GOApplication.

Tip

You can add command line arguments to Gate One by calling tornado.options.define() anywhere in your application's global namespace. This works because the define() function registers options in Gate One's global namespace (as tornado.options.options) and Gate One imports application modules before it evaluates command line arguments.

initialize()[source]

Called by ApplicationWebSocket.open() after __init__(). GOApplications can override this function to perform their own actions when the Application is initialized (happens just before the WebSocket is opened).

open()[source]

Called by ApplicationWebSocket.open() after the WebSocket is opened. GOApplications can override this function to perform their own actions when the WebSocket is opened.

on_close()[source]

Called by ApplicationWebSocket.on_close() after the WebSocket is closed. GOApplications can override this function to perform their own actions when the WebSocket is closed.

add_handler(pattern, handler, **kwargs)[source]

Adds the given handler (tornado.web.RequestHandler) to the Tornado Application (self.ws.application) to handle URLs matching pattern. If given, kwargs will be added to the tornado.web.URLSpec when the complete handler is assembled.

Note

If the pattern does not start with the configured url_prefix it will be automatically prepended.

add_timeout(timeout, func)[source]

A convenience function that calls the given func after timeout using self.io_loop.add_timeout() (which uses tornado.ioloop.IOLoop.add_timeout()).

The given timeout may be a datetime.timedelta or a string compatible with utils.convert_to_timedelta such as, "5s" or "10m".

class gateone.core.server.ApplicationWebSocket(application, request, **kwargs)[source]

The main WebSocket interface for Gate One, this class is setup to call WebSocket 'actions' which are methods registered in self.actions. Methods that are registered this way will be exposed and directly callable over the WebSocket.

classmethod file_checker()[source]

A Tornado.IOLoop.PeriodicCallback that regularly checks all files registered in the ApplicationWebSocket.watched_files dict for changes.

If changes are detected the corresponding function(s) in ApplicationWebSocket.file_update_funcs will be called.

classmethod watch_file(path, func)[source]

A classmethod that registers the given file path and func in ApplicationWebSocket.watched_files and ApplicationWebSocket.file_update_funcs, respectively. The given func will be called (by ApplicationWebSocket.file_checker) whenever the file at path is modified.

classmethod load_prefs()[source]

Loads all of Gate One's settings from options.settings_dir into cls.prefs.

Note

This classmethod gets called automatically whenever a change is detected inside Gate One's settings_dir.

classmethod broadcast_file_update()[source]

Called when there's an update to the broadcast_file (e.g. <session_dir>/broadcast); broadcasts its contents to all connected users. The message will be displayed via the GateOne.Visual.displayMessage() function at the client and can be formatted with HTML. For this reason it is important to strictly control write access to the broadcast file.

Note

Under normal circumstances only root (or the owner of the gateone.py process) can enter and/or write to files inside Gate One's session_dir directory.

The path to the broadcast file can be configured via the broadcast_file setting which can be placed anywhere under the 'gateone' application/scope (e.g. inside 10server.conf). The setting isn't there by default but you can simply add it if you wish:

"broadcast_file": "/whatever/path/you/want/broadcast"

Tip

Want to broadcast a message to all the users currently connected to Gate One? Just sudo echo "your message" > /tmp/gateone/broadcast.

initialize(apps=None, **kwargs)[source]

This gets called by the Tornado framework when ApplicationWebSocket is instantiated. It will be passed the list of apps (Gate One applications) that are assigned inside the GOApplication object. These GOApplication`s will be instantiated and stored in `self.apps.

These apps will be mutated in-place so that self will refer to the current instance of ApplicationWebSocket. Kind of like a dynamic mixin.

send_extra()[source]

Sends any extra JS/CSS files placed in Gate One's 'static/extra' directory. Can be useful if you want to use Gate One's file synchronization and caching capabilities in your app.

Note

You may have to create the 'static/extra' directory before putting files in there.

allow_draft76()[source]

By overriding this function we're allowing the older version of the WebSockets protocol. As long as communications happens over SSL there shouldn't be any security concerns with this. This is mostly to support iOS Safari.

get_current_user()[source]

Mostly identical to the function of the same name in MainHandler. The difference being that when API authentication is enabled the WebSocket will expect and perform its own auth of the client.

write_binary(message)[source]

Writes the given message to the WebSocket in binary mode (opcode 0x02). Binary WebSocket messages are handled differently from regular messages at the client (they use a completely different 'action' mechanism). For more information see the JavaScript developer documentation.

check_origin(origin)[source]

Checks if the given origin matches what's been set in Gate One's "origins" setting (usually in 10server.conf). The origin will first be checked for an exact match in the "origins" setting but if that fails each valid origin will be evaluated as a regular expression (if it's not a valid hostname) and the given origin will be checked against that.

Returns True if origin is valid.

Note

If '*' is in the "origins" setting (anywhere) all origins will be allowed.

open()[source]

Called when a new WebSocket is opened. Will deny access to any origin that is not defined in self.settings['origin']. Also sends any relevant localization data (JavaScript) to the client and calls the open() method of any and all enabled Applications.

This method kicks off the process that sends keepalive pings/checks to the client (A PeriodicCallback set as self.pinger).

This method also sets the following instance attributes:

  • self.client_id: Unique identifier for this instance.
  • self.base_url: The base URL (e.g. https://foo.com/gateone/) used to access Gate One.

Triggers the go:open event.

Note

self.settings comes from the Tornado framework and includes most command line arguments and the settings from the settings_dir that fall under the "gateone" scope. It is not the same thing as self.prefs which includes all of Gate One's settings (including settings for other applications and scopes).

on_message(message)[source]

Called when we receive a message from the client. Performs some basic validation of the message, decodes it (JSON), and finally calls an appropriate WebSocket action (registered method) with the message contents.

on_close()[source]

Called when the client terminates the connection. Also calls the on_close() method of any and all enabled Applications.

Triggers the go:close event.

on_pong(timestamp)[source]

Records the latency of clients (from the server's perspective) via a log message.

Note

This is the pong specified in the WebSocket protocol itself. The pong method is a Gate One-specific implementation.

pong(timestamp)[source]

Attached to the go:ping WebSocket action; responds to a client's ping by returning the value (timestamp) that was sent. This allows the client to measure the round-trip time of the WebSocket.

Note

This is a WebSocket action specific to Gate One. It

api_auth(auth_obj)[source]

If the auth_obj dict validates, returns the user dict and sets self.current_user. If it doesn't validate, returns False.

This function also takes care of creating the user's directory if it does not exist and creating/updating the user's 'session' file (which just stores metadata related to their session).

Example usage:

auth_obj = {
    'api_key': 'MjkwYzc3MDI2MjhhNGZkNDg1MjJkODgyYjBmN2MyMTM4M',
    'upn': 'joe@company.com',
    'timestamp': '1323391717238',
    'signature': <gibberish>,
    'signature_method': 'HMAC-SHA1',
    'api_version': '1.0'
}
result = self.api_auth(auth_obj)

See also

API Authentication documentation.

Here's a rundown of the required auth_obj parameters:

api_key:The first half of what gets generated when you run gateone --new_api_key (the other half is the secret).
upn:The userPrincipalName (aka username) of the user being authenticated.
timestamp:A 13-digit "time since the epoch" JavaScript-style timestamp. Both integers and strings are accepted. Example JavaScript: var timestamp = new Date().getTime()
signature:The HMAC signature of the combined api_key, upn, and timestamp; hashed using the secret associated with the given api_key.
signature_method:
 The hashing algorithm used to create the signature. Currently this must be one of "HMAC-SHA1", "HMAC-SHA256", "HMAC-SHA384", or "HMAC-SHA512"
api_version:Which version of the authentication API to use when performing authentication. Currently the only supported version is '1.0'.

Note

Any additional key/value pairs that are included in the auth_obj will be assigned to the self.current_user object. So if you're embedding Gate One and wish to associate extra metadata with the user you may do so via the API authentication process.

authenticate(settings)[source]

Authenticates the client by first trying to use the 'gateone_user' cookie or if Gate One is configured to use API authentication it will use settings['auth']. Additionally, it will accept settings['container'] and settings['prefix'] to apply those to the equivalent properties (self.container and self.prefix).

If settings['url'] is provided it will be used to update self.base_url (so that we can correct for situations where Gate One is running behind a reverse proxy with a different protocol/URL than what the user used to connect).

Note

'container' refers to the element on which Gate One was initialized at the client (e.g. #gateone). 'prefix' refers to the string that will be prepended to all Gate One element IDs when added to the web page (to avoid namespace conflicts). Both these values are only used when generating CSS templates.

If settings['location'] is something other than 'default' all new application instances will be associated with the given (string) value. These applications will be treated separately so they can exist in a different browser tab/window.

Triggers the go:authenticate event.

_start_session_watcher(restart=False)[source]

Starts up the SESSION_WATCHER (assigned to that global) PeriodicCallback that regularly checks for user sessions that have timed out via the timeout_sessions() function and cleans them up (shuts down associated processes).

The interval in which it performs this check is controlled via the session_timeout_check_interval setting. This setting is not included in Gate One's 10server.conf by default but can be added if needed to override the default value of 30 seconds. Example:

{
    "*": {
        "gateone": {
            "session_timeout_check_interval": "30s"
        }
    }
}
_start_cleaner()[source]

Starts up the CLEANER (assigned to that global) PeriodicCallback that regularly checks for and deletes expired user logs (e.g. terminal session logs or anything in the <user_dir>/<user>/logs dir) and old session directories via the cleanup_user_logs() and cleanup_old_sessions() functions.

The interval in which it performs this check is controlled via the cleanup_interval setting. This setting is not included in Gate One's 10server.conf by default but can be added if needed to override the default value of 5 minutes. Example:

{
    "*": {
        "gateone": {
            "cleanup_interval": "5m"
        }
    }
}
_start_file_watcher()[source]

Starts up the ApplicationWebSocket.file_watcher PeriodicCallback (which regularly calls ApplicationWebSocket.file_checker() and immediately starts it watching the broadcast file for changes (if not already watching it).

The path to the broadcast file defaults to 'settings_dir/broadcast' but can be changed via the 'broadcast_file' setting. This setting is not included in Gate One's 10server.conf by default but can be added if needed to overrided the default value. Example:

{
    "*": {
        "gateone": {
            "broadcast_file": "/some/path/to/broadcast"
        }
    }
}

Tip

You can send messages to all users currently connected to the Gate One server by writing text to the broadcast file. Example: sudo echo "Server will be rebooted as part of regularly scheduled maintenance in 5 minutes.  Pleas save your work." > /tmp/gateone/broadcast

The interval in which it performs this check is controlled via the file_check_interval setting. This setting is not included in Gate One's 10server.conf by default but can be added if needed to override the default value of 5 seconds. Example:

{
    "*": {
        "gateone": {
            "file_check_interval": "5s"
        }
    }
}
list_applications()[source]

Sends a message to the client indiciating which applications and sub-applications are available to the user.

Note

What's the difference between an "application" and a "sub-application"? An "application" is a GOApplication like app_terminal.TerminalApplication while a "sub-application" would be something like "SSH" or "nethack" which runs inside the parent application.

render_style(style_path, force=False, **kwargs)[source]

Renders the CSS template at style_path using kwargs and returns the path to the rendered result. If the given style has already been rendered the existing cache path will be returned.

If force is True the stylesheet will be rendered even if it already exists (cached).

This method also cleans up older versions of the same rendered template.

get_theme(settings)[source]

Sends the theme stylesheets matching the properties specified in settings to the client. settings must contain the following:

  • container - The element Gate One resides in (e.g. 'gateone')
  • prefix - The string being used to prefix all elements (e.g. 'go_')
  • theme - The name of the CSS theme to be retrieved.

Note

This will send the theme files for all applications and plugins that have a matching stylesheet in their 'templates' directory.

cache_cleanup(message)[source]

Attached to the go:cache_cleanup WebSocket action; rifles through the given list of message['filenames'] from the client and sends a go:cache_expired WebSocket action to the client with a list of files that no longer exist in self.file_cache (so it can clean them up).

file_request(files_or_hash, use_client_cache=True)[source]

Attached to the go:file_request WebSocket action; minifies, caches, and finally sends the requested file to the client. If use_client_cache is False the client will be instructed not to cache the file. Example message from the client requesting a file:

GateOne.ws.send(JSON.stringify({
    'go:file_request': {'some_file.js'}}));

Note

In reality 'some_file.js' will be a unique/unguessable hash.

Optionally, files_or_hash may be given as a list or tuple and all the requested files will be sent.

Files will be cached after being minified until a file is modified or Gate One is restarted.

If the slimit module is installed JavaScript files will be minified before being sent to the client.

If the cssmin module is installed CSS files will be minified before being sent to the client.

send_file(filepath, kind='misc', **metadata)[source]

Tells the client to perform a sync of the file at the given filepath. The kind should only be one of 'html' or 'misc' for HTML templates and everything else, respectively.

Any additional keyword arguments provided via metadata will be stored in the client-side fileCache database.

Note

This kind of file sending always uses the client-side cache.

send_js_or_css(paths_or_fileobj, kind, element_id=None, requires=None, media='screen', filename=None, force=False)[source]

Initiates a file synchronization of the given paths_or_fileobj with the client to ensure it has the latest version of the file(s).

The kind argument must be one of 'js' or 'css' to indicate JavaScript or CSS, respectively.

Optionally, element_id may be provided which will be assigned to the <script> or <style> tag that winds up being created (only works with single files).

Optionally, a requires string or list/tuple may be given which will ensure that the given file gets loaded after any dependencies.

Optionally, a media string may be provided to specify the 'media=' value when creating a <style> tag to hold the given CSS.

Optionally, a filename string may be provided which will be used instead of the name of the actual file when file synchronization occurs. This is useful for multi-stage processes (e.g. rendering templates) where you wish to preserve the original filename. Just be aware that if you do this the given filename must be unique.

If force is True the file will be synchronized regardless of the 'send_js' or 'send_css' settings in your global Gate One settings.

send_js(path, **kwargs)[source]

A shortcut for self.send_js_or_css(path, 'js', **kwargs).

send_css(path, **kwargs)[source]

A shortcut for self.send_js_or_css(path, 'css', **kwargs)

wrap_and_send_js(js_path, exports={}, **kwargs)[source]

Wraps the JavaScript code at js_path in a (JavaScript) sandbox which exports whatever global variables are provided via exports then minifies, caches, and sends the result to the client.

The provided kwargs will be passed to the ApplicationWebSocket.send_js method.

The exports dict needs to be in the following format:

exports = {
    "global": "export_name"
}

For example, if you wanted to use underscore.js but didn't want to overwrite the global _ variable (if already being used by a parent web application):

exports = {"_": "GateOne._"}
self.wrap_and_send_js('/path/to/underscore.js', exports)

This would result in the "_" global being exported as "GateOne._". In other words, this is what will end up at the bottom of the wrapped JavaScript just before the end of the sandbox:

window.GateOne._ = _;

This method should make it easy to include any given JavaScript library without having to worry (as much) about namespace conflicts.

Note

You don't have to prefix your export with 'GateOne'. You can export the global with whatever name you like.

render_and_send_css(css_path, element_id=None, media='screen', **kwargs)[source]

Renders, caches (in the cache_dir), and sends a stylesheet template at the given css_path. The template will be rendered with the following keyword arguments:

container = self.container
prefix = self.prefix
url_prefix = self.settings['url_prefix']
**kwargs

Returns the path to the rendered template.

Note

If you want to serve Gate One's CSS via a different mechanism (e.g. nginx) this functionality can be completely disabled by adding "send_css": false to gateone/settings/10server.conf

send_plugin_static_files(entry_point, requires=None)[source]

Sends all plugin .js and .css files to the client that exist inside the /static/ directory for given entry_point. The policies that apply to the current user will be used to determine whether and which static files will be sent.

If requires is given it will be passed along to self.send_js().

Note

If you want to serve Gate One's JavaScript via a different mechanism (e.g. nginx) this functionality can be completely disabled by adding "send_js": false to gateone/settings/10server.conf

enumerate_themes()[source]

Returns a JSON-encoded object containing the installed themes and text color schemes.

set_locales(locales)[source]

Attached to the 'go:set_locales` WebSocket action; sets self.user_locales to the given locales and calls ApplicationWebSocket.send_js_translation to deliver the best-match JSON-encoded translation to the client (if available).

The locales argument may be a string or a list. If a string, self.user_locales will be set to a list with the given locale as the first (and only) item. If locales is a list self.user_locales will simply be replaced.

send_js_translation(package='gateone', path=None, locales=None)[source]

Sends a message to the client containing a JSON-encoded table of strings that have been translated to the user's locale. The translation file will be retrieved via the pkg_resources.resource_string function using the given package with a default path (if not given) of, '/i18n/{locale}/LC_MESSAGES/gateone_js.json'. For example:

self.send_js_translation(package="gateone.plugins.myplugin")

Would result in the pkg_resources.resource_string function being called like so:

path = '/i18n/{locale}/LC_MESSAGES/gateone_js.json'.format(
    locale=locale)
translation = pkg_resources.resource_string(
    "gateone.plugins.myplugin", path)

If providing a custom path be sure it includes the '{locale}' portion so the correct translation will be chosen. As an example, your plugin might store locales inside a 'translations' directory with each locale JSON translation file named, '<locale>/myplugin.json'. The correct path for that would be: '/translations/{locale}/myplugin.json'

If no locales (may be list or string) is given the self.user_locales variable will be used.

This method will attempt to find the closest locale if no direct match can be found. For example, if locales=="fr" the 'fr_FR' locale would be used. If locales is a list, the first matching locale will be used.

Note

Translation files must be the result of a pojson /path/to/translation.po conversion.

send_message(message, session=None, upn=None)[source]

Sends the given message to the client using the go:user_message WebSocket action at the currently-connected client.

If upn is provided the message will be sent to all users with a matching 'upn' value.

If session is provided the message will be sent to all users with a matching session ID. This is useful in situations where all users share the same 'upn' (i.e. ANONYMOUS).

if upn is 'AUTHENTICATED' all users will get the message.

classmethod _deliver(message, upn='AUTHENTICATED', session=None)[source]

Writes the given message (string) to all users matching upn using the write_message() function. If upn is not provided or is "AUTHENTICATED", will send the message to all users.

Alternatively a session ID may be specified instead of a upn. This is useful when more than one user shares a UPN (i.e. ANONYMOUS).

classmethod _list_connected_users()[source]

Returns a tuple of user objects representing the users that are currently connected (and authenticated) to this Gate One server.

license_info()[source]

Returns the contents of the __license_info__ dict to the client as a JSON-encoded message.

class gateone.core.server.ErrorHandler(application, request, status_code)[source]

Generates an error response with status_code for all requests.

gateone.core.server.validate_authobj(args=['/usr/bin/sphinx-build', '-b', 'html', '-v', '-T', '-d', 'build/doctrees', 'source', 'build/html'])[source]

Handles the 'validate_authobj' CLI command. Takes a JSON object as the only argument (must be inside single quotes) and validates the singature using the same mechanism as ApplicationWebSocket.api_auth. Example usage:

user@modern-host:~ $ gateone validate_authobj '{"upn": "jdoe@company.com", "signature_method": "HMAC-SHA1", "timestamp": "1409266590093", "signature": "004464e27db90180a4b87b50b00dd77420052b6d", "api_key": "NGQxNTVjZWEzMmM1NDBmNGI5MzYwNTM3ZDY0MzZiNTczY", "api_version": "1.0"}'
API Authentication Successful!
gateone.core.server.install_license(args=['/usr/bin/sphinx-build', '-b', 'html', '-v', '-T', '-d', 'build/doctrees', 'source', 'build/html'])[source]

Handles the 'install_license' CLI command. Just installs the license at the path given via sys.argv[1] (first argument after the 'install_license' command).

gateone.core.server.validate_licenses(licenses)[source]

Given a licenses dict, logs and removes any licenses that have expired or have invalid signatures. It then sets the __license_info__ global with the updated dict. Example license dict:

{
    "11252c41-d3cd-45b7-929b-4a3baedcc152": {
        "product": "gateone",
        "customer": "ACME, Inc",
        "users": 250,
        "expires": 1441071222,
        "license_format": "1.0",
        "signature": "<gobbledygook>",
        "signature_method": "secp256k1"
    }
}

Note

Signatures will be validated against Liftoff Software's public key.