Skip to main content
Version: 2.0

Module Dependencies

GraphGrid modules can be individually stopped and started. Some modules will not perform as expected without other modules and services running. This page provides information about each GraphGrid module and their dependencies.

Modules and services​

GraphGrid Modules

Config
Security
Files
Fuze
Manager
PDF
Publish
Search
Curator
NLP
Payment
ML

Services

Elasticsearch
MinIO
RabbitMQ
Redis
OpenLDAP
PHPLDAPAdmin
Nginx
ONgDB
Postgres
Kibana
Logstash
Logspout
Apm-server
Spark-master
Spark-worker (1,2)
Airflow
Airflow-flower
Airflow-worker
airflow-scheduler
Airflow-init
Socat

GraphGrid Module dependency​

Config​

GraphGrid Config requires the following services to run:

  • Nginx
  • RabbitMQ
  • Redis
  • ONgDB
  • GraphGrid Security*
  • GraphGrid Config

Config Startup Command​

Starting up Config with the GraphGrid start command will bring up the following services:

(base) user@desktop graphgrid-ai-2.0.0-SNAPSHOT % ./bin/graphgrid start config
Starting GraphGrid services
docker-compose up -d config
Creating network "graphgrid" with driver "bridge"
Creating graphgrid-ai-200-snapshot_nginx_1    ... done
Creating graphgrid-ai-200-snapshot_rabbitmq_1 ... done
Creating graphgrid-ai-200-snapshot_redis_1    ... done
Creating graphgrid-ai-200-snapshot_ongdb_1    ... done
Creating graphgrid-ai-200-snapshot_config_1   ... done
note

*GraphGrid Security does not come up with the ./bin/graphgrid start config command. GraphGrid Config has its own security controls to provide modules and service direct access. Users should startup GraphGrid Security if they need user authentication and authorization. Without GraphGrid Security running, a 500 error is thrown when running Config endpoints that require an authorization token. Start GraphGrid Security the following command: ./bin/graphgrid start security

Curator​

GraphGrid Curator requires the following services to run:

  • RabbitMQ
  • Nginx
  • Redis
  • Postgres
  • OpenLDAP
  • ONgDB
  • GraphGrid Config
  • GraphGrid Security
  • GraphGrid Search
  • GraphGrid Manager
  • GraphGrid Curator

Curator Startup Command​

Starting up Curator with the GraphGrid start command will bring up the following services:

(base) user@desktop graphgrid-ai-2.0.0-SNAPSHOT % ./bin/graphgrid start curator
Starting GraphGrid services
docker-compose up -d curator
Creating network "graphgrid" with driver "bridge"
Creating graphgrid-ai-200-snapshot_rabbitmq_1 ... done
Creating graphgrid-ai-200-snapshot_nginx_1    ... done
Creating graphgrid-ai-200-snapshot_redis_1    ... done
Creating graphgrid-ai-200-snapshot_postgres_1 ... done
Creating graphgrid-ai-200-snapshot_openldap_1 ... done
Creating graphgrid-ai-200-snapshot_ongdb_1    ... done
Creating graphgrid-ai-200-snapshot_config_1   ... done
Creating graphgrid-ai-200-snapshot_security_1 ... done
Creating graphgrid-ai-200-snapshot_manager_1  ... done
Creating graphgrid-ai-200-snapshot_search_1   ... done
Creating graphgrid-ai-200-snapshot_curator_1  ... done
note

GraphGrid Manager sets up event triggers that effect the graph, including assigning each node a unique identifier called a grn. The Curator Dashboard will not work as expected without the triggers set up through Manager. Do not stop or disable Manager while using Curator.

Files​

GraphGrid Files requires the following services to run:

  • OpenLDAP
  • Postgres
  • Nginx
  • MinIO
  • RabbitMQ
  • Redis
  • ONgDB
  • GraphGrid Config
  • GraphGrid Security
  • GraphGrid Files

Files Startup Command​

Starting up Files with the GraphGrid start command will bring up the following services:

(base) user@desktop graphgrid-ai-2.0.0-SNAPSHOT % ./bin/graphgrid start files
Starting GraphGrid services
docker-compose up -d files
Creating network "graphgrid" with driver "bridge"
Creating graphgrid-ai-200-snapshot_openldap_1 ... done
Creating graphgrid-ai-200-snapshot_postgres_1 ... done
Creating graphgrid-ai-200-snapshot_nginx_1    ... done
Creating graphgrid-ai-200-snapshot_minio_1    ... done
Creating graphgrid-ai-200-snapshot_rabbitmq_1 ... done
Creating graphgrid-ai-200-snapshot_redis_1    ... done
Creating graphgrid-ai-200-snapshot_ongdb_1    ... done
Creating graphgrid-ai-200-snapshot_config_1   ... done
Creating graphgrid-ai-200-snapshot_security_1 ... done
Creating graphgrid-ai-200-snapshot_files_1    ... done

Fuze​

GraphGrid Fuze requires the following services to run:

  • Nginx
  • Postgres
  • RabbitMQ
  • OpenLDAP
  • Redis
  • ONgDB
  • GraphGrid Config
  • GraphGrid Security
  • GraphGrid Fuze

Fuze Startup Command​

Starting up Fuze with the GraphGrid start command will bring up the following services:

(base) user@desktop graphgrid-ai-2.0.0-SNAPSHOT % ./bin/graphgrid start fuze
Starting GraphGrid services
docker-compose up -d fuze
Creating network "graphgrid" with driver "bridge"
Creating graphgrid-ai-200-snapshot_nginx_1    ... done
Creating graphgrid-ai-200-snapshot_postgres_1 ... done
Creating graphgrid-ai-200-snapshot_rabbitmq_1 ... done
Creating graphgrid-ai-200-snapshot_openldap_1 ... done
Creating graphgrid-ai-200-snapshot_redis_1    ... done
Creating graphgrid-ai-200-snapshot_ongdb_1    ... done
Creating graphgrid-ai-200-snapshot_config_1   ... done
Creating graphgrid-ai-200-snapshot_security_1 ... done
Creating graphgrid-ai-200-snapshot_fuze_1     ... done

NLP​

GraphGrid NLP requires the following services to run:

  • Postgres
  • OpenLDAP
  • Nginx
  • Redis
  • RabbitMQ
  • MinIO
  • ONgDB
  • GraphGrid Config
  • GraphGrid Security
  • GraphGrid Fuze
  • GraphGrid NLP

NLP Startup Command​

Starting up NLP with the GraphGrid start command will bring up the following services:

(base) user@desktop graphgrid-ai-2.0.0-SNAPSHOT % ./bin/graphgrid start nlp
Starting GraphGrid services
docker-compose up -d nlp
Creating network "graphgrid" with driver "bridge"
Creating graphgrid-ai-200-snapshot_postgres_1 ... done
Creating graphgrid-ai-200-snapshot_openldap_1 ... done
Creating graphgrid-ai-200-snapshot_nginx_1    ... done
Creating graphgrid-ai-200-snapshot_redis_1    ... done
Creating graphgrid-ai-200-snapshot_rabbitmq_1 ... done
Creating graphgrid-ai-200-snapshot_minio_1    ... done
Creating graphgrid-ai-200-snapshot_ongdb_1    ... done
Creating graphgrid-ai-200-snapshot_config_1   ... done
Creating graphgrid-ai-200-snapshot_security_1 ... done
Creating graphgrid-ai-200-snapshot_fuze_1     ... done
Creating graphgrid-ai-200-snapshot_nlp_1      ... done
note

Data extraction relies on triggers created by Fuze. Do not stop or disable Fuze while running NLP individually.

If the following error is given while attempting data extraction, Fuze is not running and should be started:

[SimpleAsyncTaskExecutor-1]    ERROR c.g.sdk.core.GraphGridHttpClient     - Error processing request: com.graphgrid.sdk.model.neo4jWriter.Neo4jWriterRequest@5f342f54[endpoint=<http://fuze:8080/1.0/fuze/write,serviceUrl>=<null>,body=com.graphgrid.sdk.model.neo4jWriter.TransactionRequest@5277db20,headers={Authorization=Bearer 0495f13d-fa6f-4757-bb23-33367b64bcd4, graphgrid-user-token=null},customQueryParameters=<null>,requestHandler=<null>,responseHandler=<null>,requestAuthMethod=com.graphgrid.sdk.core.security.ClientCredentialsTokenRequest@24ad98d2]
nlp_1                | 2023-10-09 15:06:37,210 [SimpleAsyncTaskExecutor-1]    ERROR c.g.sdk.core.GraphGridHttpClient     - Error processing request: com.graphgrid.sdk.model.neo4jWriter.Neo4jWriterRequest@5f342f54[endpoint=<http://fuze:8080/1.0/fuze/write,serviceUrl>=<null>,body=com.graphgrid.sdk.model.neo4jWriter.TransactionRequest@5277db20,headers={Authorization=Bearer 0495f13d-fa6f-4757-bb23-33367b64bcd4, graphgrid-user-token=null},customQueryParameters=<null>,requestHandler=<null>,responseHandler=<null>,requestAuthMethod=com.graphgrid.sdk.core.security.ClientCredentialsTokenRequest@24ad98d2]

Manager​

GraphGrid Manager requires the following services to run:

  • OpenLDAP
  • Redis
  • Nginx
  • Elasticsearch
  • Postgres
  • RabbitMQ
  • ONgDB
  • GraphGrid Config
  • GraphGrid Security
  • GraphGrid Manager

Manager Startup Command​

Starting up Manager with the GraphGrid start command will bring up the following services:

(base) user@desktop graphgrid-ai-2.0.0-SNAPSHOT % ./bin/graphgrid start manager
Starting GraphGrid services
docker-compose up -d manager
Creating network "graphgrid" with driver "bridge"
Creating graphgrid-ai-200-snapshot_openldap_1      ... done
Creating graphgrid-ai-200-snapshot_redis_1         ... done
Creating graphgrid-ai-200-snapshot_nginx_1         ... done
Creating graphgrid-ai-200-snapshot_elasticsearch_1 ... done
Creating graphgrid-ai-200-snapshot_postgres_1      ... done
Creating graphgrid-ai-200-snapshot_rabbitmq_1      ... done
Creating graphgrid-ai-200-snapshot_ongdb_1         ... done
Creating graphgrid-ai-200-snapshot_config_1        ... done
Creating graphgrid-ai-200-snapshot_security_1      ... done
Creating graphgrid-ai-200-snapshot_manager_1       ... done

ML​

GraphGrid ML requires the following services to run:

  • Postgres
  • OpenLDAP
  • Nginx
  • Redis
  • RabbitMQ
  • MinIO
  • ONgDB
  • Spark
  • Airflow
  • GraphGrid Config
  • GraphGrid Security
  • GraphGrid ML

ML Startup Command​

note

The GraphGrid ML module and dependent services are disabled by default. ML must be enabled before using the start command. Use the following command to enable ML: 

./bin/graphgrid enable ml

Starting up ML without enabling it first will produce the following error message:

(base) user@desktop graphgrid-ai-skeleton-2.0.0-SNAPSHOT 2 % ./bin/graphgrid start ml 
Starting GraphGrid services
docker-compose up -d ml
ERROR: Service "spark-master" was pulled in as a dependency of service "spark-worker-1" but is not enabled by the active profiles. You may fix this by adding a common profile to "spark-master" and "spark-worker-1".

Starting up ML with the GraphGrid start command will bring up the following services:

(base) user@desktop graphgrid-ai-skeleton-2.0.0-SNAPSHOT 2 % ./bin/graphgrid start ml 
Starting GraphGrid services
docker-compose up -d ml
Creating network "graphgrid" with driver "bridge"
Creating graphgrid-ai-skeleton-200-snapshot2_spark-master_1  ... done
Creating graphgrid-ai-skeleton-200-snapshot2_postgres_1      ... done
Creating graphgrid-ai-skeleton-200-snapshot2_rabbitmq_1       ... done
Creating graphgrid-ai-skeleton-200-snapshot2_elasticsearch_1 ... done
Creating graphgrid-ai-skeleton-200-snapshot2_minio_1          ... done
Creating graphgrid-ai-skeleton-200-snapshot2_openldap_1       ... done
Creating graphgrid-ai-skeleton-200-snapshot2_redis_1          ... done
Creating graphgrid-ai-skeleton-200-snapshot2_nginx_1          ... done
Creating graphgrid-ai-skeleton-200-snapshot2_spark-worker-2_1 ... done
Creating graphgrid-ai-skeleton-200-snapshot2_spark-worker-1_1 ... done
Creating graphgrid-ai-skeleton-200-snapshot2_ongdb_1          ... done
Creating graphgrid-ai-skeleton-200-snapshot2_config_1         ... done
Creating graphgrid-ai-skeleton-200-snapshot2_security_1       ... done
Creating graphgrid-ai-skeleton-200-snapshot2_manager_1        ... done
Creating graphgrid-ai-skeleton-200-snapshot2_ml_1             ... done

PDF​

GraphGrid PDF requires the following services to run:

  • MinIO
  • OpenLDAP
  • Postgres
  • Nginx
  • Redis
  • RabbitMQ
  • ONgDB
  • GraphGrid Config
  • GraphGrid Security
  • GraphGrid PDF

PDF Startup Command​

Starting up PDF with the GraphGrid start command will bring up the following services:

(base) user@desktop graphgrid-ai-2.0.0-SNAPSHOT % ./bin/graphgrid start pdf
Starting GraphGrid services
docker-compose up -d pdf
Creating network "graphgrid" with driver "bridge"
Creating graphgrid-ai-200-snapshot_minio_1    ... done
Creating graphgrid-ai-200-snapshot_openldap_1 ... done
Creating graphgrid-ai-200-snapshot_postgres_1 ... done
Creating graphgrid-ai-200-snapshot_nginx_1    ... done
Creating graphgrid-ai-200-snapshot_redis_1    ... done
Creating graphgrid-ai-200-snapshot_rabbitmq_1 ... done
Creating graphgrid-ai-200-snapshot_ongdb_1    ... done
Creating graphgrid-ai-200-snapshot_config_1   ... done
Creating graphgrid-ai-200-snapshot_security_1 ... done
Creating graphgrid-ai-200-snapshot_pdf_1      ... done

Publish​

GraphGrid Publish requires the following services to run:

  • OpenLDAP
  • Postgres
  • Nginx
  • RabbitMQ
  • MinIO
  • Redis
  • ONgDB
  • GraphGrid Config
  • GraphGrid Security
  • GraphGrid Publish

Publish Startup Command​

Starting up Publish with the GraphGrid start command will bring up the following services:

(base) user@desktop graphgrid-ai-2.0.0-SNAPSHOT % ./bin/graphgrid start publish
Starting GraphGrid services
docker-compose up -d publish
Creating network "graphgrid" with driver "bridge"
Creating graphgrid-ai-200-snapshot_openldap_1 ... done
Creating graphgrid-ai-200-snapshot_postgres_1 ... done
Creating graphgrid-ai-200-snapshot_nginx_1    ... done
Creating graphgrid-ai-200-snapshot_rabbitmq_1 ... done
Creating graphgrid-ai-200-snapshot_minio_1    ... done
Creating graphgrid-ai-200-snapshot_redis_1    ... done
Creating graphgrid-ai-200-snapshot_ongdb_1    ... done
Creating graphgrid-ai-200-snapshot_config_1   ... done
Creating graphgrid-ai-200-snapshot_security_1 ... done
Creating graphgrid-ai-200-snapshot_publish_1  ... done

GraphGrid Search requires the following services to run:

  • Postgres
  • OpenLDAP
  • Redis
  • RabbitMQ
  • Elasticsearch
  • Nginx
  • ONgDB
  • GraphGrid Config
  • GraphGrid Security
  • GraphGrid Search

Search Startup Command​

Starting up Search with the GraphGrid start command will bring up the following services:

(base) user@desktop graphgrid-ai-2.0.0-SNAPSHOT % ./bin/graphgrid start search
Starting GraphGrid services
docker-compose up -d search
Creating graphgrid-ai-200-snapshot_postgres_1      ... done
Creating graphgrid-ai-200-snapshot_openldap_1      ... done
Creating graphgrid-ai-200-snapshot_redis_1         ... done
Creating graphgrid-ai-200-snapshot_rabbitmq_1      ... done
Creating graphgrid-ai-200-snapshot_elasticsearch_1 ... done
Creating graphgrid-ai-200-snapshot_nginx_1         ... done
Creating graphgrid-ai-200-snapshot_ongdb_1         ... done
Creating graphgrid-ai-200-snapshot_config_1        ... done
Creating graphgrid-ai-200-snapshot_security_1      ... done
Creating graphgrid-ai-200-snapshot_search_1        ... done