# Modules

# Overview

Modules are how application builders share configured connections to external resources. A configured CloudWright module provides configure-once, use-anywhere access to a specific third-party service, database, API, or cloud-native tool (for example, S3 or BigQuery).

You can think of a module as a Python package which comes bundled with configurations, access credentials, and network awareness so you can use it in a new application without needing any boilerplate. This guide describes how to create, share, and use modules in your applications.

# Using a module

There are two steps to using a module in an application: (1) attaching the module to the application through the module sidebar, and (2) getting a reference to the module via CloudWright.get_module and calling it in your application

First, navigate to the 'Modules' tab in the right sidebar, and click 'Edit'. This takes you to the module selector view:

Edit Modules

All modules available to use in an application will be shown in the left sidebar. To be available, a module must be:

  • In the same team as the application (or of the application is a personal appliation, in any team you belong to or a personal module)
  • In a Deployment Zone compatible with your application (for details on module deployment zones, see creating a module)

From the left column, select the modules you want to include in this application, and save your changes.

CloudWright.get_module returns a code reference to the module in your application. In most cases, the module entrypoint will be an instance of a third-party library (for example, the Datadog Python API) and can be used directly:

datadog = CloudWright.get_module("datadog")
datadog.Event.create(
  title="New Event",
  text="Something happened!",
  tags=["version:1", "application:web"])

You can find details and examples about how to use a specific module through the 'Examples' link in the sidebar:

Edit Modules

or by clicking the hyperlink in the code editor (here, 'bigquery_sandbox'):

Edit Modules

The module description will show example usages, link to external documentation for full API documentation, and show other usages in your team:

Edit Modules

# Import Keys and Ids

When a new module is created from a template, the module inherits an import key based on which template it used. The import key is just a string naming the template - for example, the import key for all modules which connect to Datadog is datadog. Accessing modules via their import key makes it easy for others to read application code and know what kinds of modules are being used, and also makes it possible to copy applications and swap in different modules that do the same thing.

If an application uses multiple modules that have the same template (and thus the same import key), an application editor will need to indicate which module they wish to reference when they call CloudWright.get_module by supplying the id of the module. For specifics on importing modules in application code, see Importing Modules in Applications.

# Application images

When a CloudWright module is added to your application, behind-the-scenes, it will bundle one or more pieces of supporting infrastructure into the deployed application:

  • Configurations (for example, a database hostname)
  • Secrets for authenticating to the external resource (for example, database passwords)
  • Deployment Zone restrictions (for example, a database may be reachable only from a certain VPC network)
  • Python libraries used for accessing the module (for example, the AWS S3 library)

The libraries and configurations provided by an application's connected modules are built into the application's image.

By building configurations into the application image, dry-run executions can execute quickly and deploys are fast and lightweight. It does, however, mean that when an application's list of modules changes, it triggers an image build which may take several minutes. Dry runs will not use the new image until the build is successful.

# Visibility

The visibility of a module controls who has permission to view, use, or edit a module. Since modules grant access to sensitive resources, it is important to create modules with the appropriate visibility.

Similar to applications, there are two levels of module visibility:

  • Personal modules, which can only ever be viewed, edited, or used in your own personal applications. Personal modules are appropriate when a module has access to sensitive personal resources — for example, a module which has been granted OAuth access to your Google Docs.

  • Shared modules, which can be viewed, edited, or used in applications by an member of a specific team. Shared modules are appropriate for team-wide resources — for example, access to data in an S3 bucket.

The visibility of a module cannot be changed after it is created.

# Creating a module

To create a module, navigate to the 'Modules' tab in the left sidebar. This will show a list of the modules available in each of the teams you belong to, as well as your personal modules:

Edit Modules

To create a new module, click 'New' in the upper left. This take you to the module creation page:

Edit Modules

First, you'll need to choose the appropriate visibility, and potentially team membership, for your new module. See the visibility section for details on this decision.

Once you've chosen the visibility of the module, you'll be shown a list of module templates (for a full list of available templates, see Module Templates. These templates are all the services or tools for which CloudWright has integrations (if you don't see a module template for the service you're trying to connect to, please let us know and we'll do our best to add it).

To create the new module, you'll need to choose a template to instantiate, by clicking the 'Copy' icon from the left sidebar.

Edit Modules

The first configuration items are shared across all modules you create:

The Display Name of a module is the name of the module shown in all UI contexts (lists, selectors, etc), but not in source code. The Display Name of a module can be changed at any time.

The ID of a module is the key used to reference the module within an application's source code if the user has imported multiple modules of the same kind (that is, having the same import key). For example, if your Datadog module's ID is 'my_datadog_module', users could access your specific module by calling CloudWright.get_module("datadog", instance="my_datadog_module"). Because this ID may be embedded in application code, it cannot be changed after the module is created.

A module Description is a summary of what the module does and what it connects to. The description can be edited at any time.

A Deployment Zone Restriction can restrict usage of a module to applications published into a particular Deployment Zone. For example, if your module connects to a MySQL database on a private IP address, it does not make sense for an application in the 'CloudWright Managed' DZ to use the module. Restricting the module to the 'My Internal' DZ prevents users from accidentally constructing broken applications.

Most module templates will require (or allow optional) additional configuration items — passwords, API keys, or OAuth logins. These will be saved with the module and can be changed later. When a CloudWright Module is given secret credentials like AWS secret keys, they are encrypted and stored in a database isolated from other customers' data and are subject to team-level access controls.

Click 'Create' to save your new module — it is now available to use in applications.

# Staging Configurations

Most Modules can optionally be configured with a Staging Configuraiton. This is an alternative set of configurations which are used in Editor Dry Runs. Published Applications will always use the Production set of configurations.

For example, consider a MySQL Module:

  • The Module's Production Configuration points at our production database host.
  • The Staging Configuration points at a mirrored development database and is given a user with read-only access.

When developing an Application with this Module, say we want to test out a query we've written. To do this, we execute the code we've developed using Dry Run in the Editor. Since we're running code using Dry Run, the Module's Staging Configuration is used. If we accidentally write an expensive query, no harm is done to the production database.

The Published version of our Application will use the Production Configuration. This includes the following:

# Creating a Staging Configuration

A Staging Configuration can be added by clicking on the "Add Staging" link in the Module editor. You'll then be able to enter configurations relevant to the staging target for this module.

Staging Configuration

Modules without a Staging Configuration will use the Production Configuration for both Dry Run executions and Published executions.