Skip to main content
Version: 2.0

GraphGrid Python Template Project


The GraphGrid Python Template Project provides a generalized codebase for creating an Airflow deployment to tackle tasks which benefit from Airflow orchestration.


The GraphGrid Python Template Project requires the following software libraries:

  • helm
  • kubectl
  • minikube
  • docker

Features Overview

  • template-project command line interface
  • local-airflow command line interface
  • Basic expandable Airflow DAG
  • Testing suite
  • Linting


This project is intended to be expanded upon by employing it into a specific task (e.g. machine learning, ingest, etc.). However, the existing CLIs are the main way of using this codebase. The local-airflow CLI includes commands for deploying Airflow locally and managing the images its built upon. Whereas, the template-project CLI is a simple skeleton CLI to be built upon, but includes print, lint, and test.

The DAG and other Airflow specific functionality is tied directly to the Airflow dockerfile and Docker image. Therefore, the Airflow image needs to be rebuilt when expanding the DAG to include new tasks or new functionality. This also applies to anything related to backend Airflow functionality, e.g. where DAGs are stored, how logging functions, etc.

All other functionality is tied to the template-project image. For example, if a new invokable command is added to the template-project CLI, or the way existing commands change (e.g. the print/test/lint commands), then the template-project image needs to be updated to reflect that.

Local Airflow CLI

The local Airflow cli has multiple commands in order to facilitate local development and testing. The expected usage for the cli takes the following form: local-airflow <command> <command_args>.

This section will cover all the available commands for the local-airflow cli.


Locally deploy the Airflow cluster with all dependencies

Arguments: None


Open the minikube dashboard to debug the deployment

Arguments: None


Forwards ports to allow external local access to the Airflow

Arguments: None


Tears down the local minikube deployment

Arguments: None


Rebuilds the Airflow DAG image and redeploys with the latest image

Arguments: None


Rebuilds the template project image

Arguments: None


Build the template project image and Airflow DAG

Arguments: None

Template project CLI

The template project cli is setup to be expanded upon as it is how the DAG spins up tasks within the Kubernetes cluster. However, it is dyanmic in that it can be invoked locally outside of kubernetes to perform tasks.

The expected usage for the cli takes the following form: template-project <task> <task_args>.

This section will cover all the available commands for the template-project cli.


Prints a given message to stdout


  • message (Required): The message to print


Runs the entire test suite on the codebase and reports basic coverage

Arguments: None


Lints the entire codebase and reports a score

Arguments: None

Development and Extension

The intended means of extending and developing via the template-project is through extending the template-project CLI. For example, if the template-project is used to tackle a machine learning project, then this would include adding commands to the CLI which could build, train, and evaluate machine learning models. As this follows a basic directed workflow (i.e. building before training, and training before evaluating), this can define the DAG for this machine-learning task. Therefore, we add these invokable commands as Airflow tasks within the DAG. Then we can leverage the local-airflow CLI to deploy and test out this workflow and codebase.