This page demonstrates the process of importing data from a relational database into an Ultipa graphset.

Ultipa Transporter supports data import from MySQL, PostgreSQL, SQL Server, Oracle, and Snowflake in sql mode using a unified configuration file.
The following steps are demonstrated using MySQL.

Generate Configuration File

Execute the following command in your command line tool and select sql.

./ultipa-importer --sample

The import.sample.sql.yml file will be generated in the same directory as ultipa-importer.exe. If a import.sample.sql.yml file already exists in that directory, it will be overwritten.

Modify Configuration File

The configuration file consists of several sections. Modify the configuration file according to your needs.

# Mode options: csv/json/jsonl/rdf/graphml/bigQuery/sql/kafka/neo4j/salesforce; only one mode can be used
# SQL supports mysql/postgresSQL/sqlserver/snowflake/oracle
mode: sql

# SQL server configurations
sqlDatabase:
  # Driver: choose from mysql/postgreSQL/sqlserver/snowflake/odbc/oracle
  driver: "mysql"
  # dsn: Data Source Name
  # If a DSN is provided, it will take precedence over individual connection settings
  # If no DSN is specified, the individual settings (host, port, dbname, username, and password) will be used for connecting to the database
  # mysql dsn: "<username>:<password>@tcp(<host>:<port>)/<dbname>?net_write_timeout=6000"
  # postgreSQL dsn: "user=<username> dbname=<dbname> password=<password> host=<host> port=<port> sslmode=disable"
  # sqlserver dsn: "server=<host>,<port>;user id=<username>;password=<password>;database=<dbname>"
  # snowflake dsn: "<username>:<password>@<orgname>-<account_name>/<db_name>/<schema_name>?warehouse=<warehouse_name>"
  # odbc dsn: "DSN=<dsn>;UID=<username>;PWD=<password>"
  # oracle dsn: "oracle://<username>:<password>@<host>:<port>/<dbname>"
  dsn: ""
  # Host IP/URI
  host: "192.168.1.1"
  port: "3306"
  dbname: "trading"
  username: "root"
  password: "root"

# Ultipa server configurations
server:
  # Host IP/URI and port
  # If it is a cluster, separate hosts with commas, i.e., "<ip1>:<port1>,<ip2>:<port2>,<ip3>:<port3>"
  host: "10.11.22.33:1234"
  username: "admin"
  password: "admin12345"
  # The new or existing graphset where data will be imported
  graphset: "trading"
  # If the above graphset is new, specify the shards where it will be stored  
  shards: "1,2,3"
  # If the above graphset is new, specify the partition function (Crc32/Crc64WE/Crc64XZ/CityHash64) used for sharding
  partitionBy: "Crc32"
  # Path of the certificate file for TLS (optional)
  crt: ""

# Node configurations
nodeConfig:
    # Specify the node type (schema) that the imported nodes belong to
  - schema: "customer"
    # Write the SQL query to fetch data from the SQL database
    sql: "select * from customer"

    # properties: Map SQL query results to graph database properties
    # For each property, you can configure the following:
    ## name: The column name from the SQL query results
    ## new_name: The property name to which the column will be mapped; if unset, it defaults to the column name
    ## type: Supported types include _id, _from, _to, _ignore (to skip importing the column), and other Ultipa property types like int64, int32, float, string, etc.
    ## prefix: Add a prefix to the values of the _id, _from, or _to types; it does not apply to other types
    # If properties are not configured, the system will automatically map them based on the SQL query results
    # Columns mapped to system properties such as _id, _from or _to in the SQL results must be explicitly configured
    properties:
      - name: cust_no
        type: _id
      - name: name
        type: string
      - name: level
        type: int32
  - schema: "merchant"
    sql: "select * from merchant"
    properties:
      - name: merch_no
        type: _id
      - name: name
        type: string
      - name: type
        type: string

# Edge configurations
edgeConfig:
  - schema: "transaction"
    sql: "select * from transaction"
    properties:
      - name: trans_no
        type: string
      - name: cust_no
        type: _from
      - name: merch_no
        type: _to
      - name: time
        new_name: trans_time
        type: datetime

# Global settings
settings:
  # Define the path to output the log file
  logPath: "./logs"
  # Number of rows included in each insertion batch
  batchSize: 10000
  # Import mode supports insert/overwrite/upsert
  importMode: insert
  # Automatically create missing end nodes for edges (applicable only when importing edges)
  createNodeIfNotExist: false
  # Stops the importing process when error occurs
  stopWhenError: false
  # Set to true to automatically create missing graphset, schemas and properties
  yes: true
  # The maximum threads
  threads: 32
  # The maximum size (in MB) of each packet
  maxPacketSize: 40
  # Timezone for the timestamp values
  # timeZone: "+0200"
  # Timestamp value unit, support ms/s
  timestampUnit: s

Configuration Items

SQL server configurations

Ultipa server configurations

Node/Edge configurations

Global settings

Execute Import

The import process uses the configuration file specified by the -config parameter to import data from the relaitonal database into the target server and display it in the Ultipa graph structure.

./ultipa-importer --config import.sample.sql.yml