This page demonstrates the process of importing data from JSON file(s) into an Ultipa graphset.

Generate Configuration File

Execute the following command in your command line tool and select json.

./ultipa-importer --sample

The import.sample.json.yml file will be generated in the same directory as ultipa-importer.exe. If a import.sample.json.yml file already exists in that directory, it will be overwritten.

Modify Configuration File

The configuration file consists of several sections. Modify the configuration file according to your needs.

If SFTP is not used, remove the section involving SFTP server configurations from the configuration file below.

# Mode options: csv/json/jsonl/rdf/graphml/bigQuery/sql/kafka/neo4j/salesforce; only one mode can be used
# SQL supports mysql/postgresSQL/sqlserver/snowflake/oracle
mode: json

# Ultipa server configurations
server:
  # Host IP/URI and port
  # If it is a cluster, separate hosts with commas, i.e., "<ip1>:<port1>,<ip2>:<port2>,<ip3>:<port3>"
  host: "10.11.22.33:1234"
  username: "admin"
  password: "admin12345"
  # The new or existing graphset where data will be imported
  graphset: "trading"
  # If the above graphset is new, specify the shards where it will be stored  
  shards: "1,2,3"
  # If the above graphset is new, specify the partition function (Crc32/Crc64WE/Crc64XZ/CityHash64) used for sharding
  partitionBy: "Crc32"
  # Path of the certificate file for TLS (optional)
  crt: ""

# SFTP server configurations
sftp:
  # Host IP/URI and port
  host: "10.12.23.34:4567"
  username: "yourUsername"
  password: "yourPassword"
  # SSH Key path for SFTP (if set, password will not be used)
  key: "./my_secret"

# Node configurations
nodeConfig:
    # Specify the node type (schema) that the imported nodes belong to
  - schema: "customer"
    # Specify the file path. If SFTP is configured, provide the SFTP path
    file: "./dataset/trading/customer.json"

    # properties: Map data to graph database properties
    # For each property, you can configure the following:
    ## name: The key identifier from the JSON file
    ## new_name: The property name to which the value will be mapped; if unset, it defaults to the key
    ## type: Supported types include _id, _from, _to, and other Ultipa property types like int64, int32, float, string, etc.
    ## prefix: Add a prefix to the values of the _id, _from, or _to types; it does not apply to other types
    # All desired properties must be configured to ensure successful data mapping 
    properties:
      - name: cust_no
        type: _id
      - name: name
        type: string
      - name: level
        type: int32
  - schema: "merchant"
    file: "./dataset/trading/merchant.json"
    properties: 
      - name: merch_no
        type: _id
      - name: name
        type: string
      - name: type
        type: string

# Edge configurations
edgeConfig:
  - schema: "transaction"
    file: "./dataset/trading/transaction.json"
    properties:
      - name: trans_no
        type: string
      - name: cust_no
        type: _from
      - name: merch_no
        type: _to
      - name: time
        new_name: trans_time
        type: datetime

# Global settings
settings:
  # Define the path to output the log file
  logPath: "./logs"
  # Number of rows included in each insertion batch
  batchSize: 10000
  # Import mode supports insert/overwrite/upsert
  importMode: insert
  # Automatically create missing end nodes for edges (applicable only when importing edges)
  createNodeIfNotExist: false
  # Stops the importing process when error occurs
  stopWhenError: false
  # Set to true to automatically create missing graphset, schemas and properties
  yes: true
  # The maximum threads
  threads: 32
  # The maximum size (in MB) of each packet
  maxPacketSize: 40
  # Timezone for the timestamp values
  # timeZone: "+0200"
  # Timestamp value unit, support ms/s
  timestampUnit: s

Configuration Items

Ultipa server configurations

SFTP server configurations

Node/Edge configurations

Global settings

Execute Import

The import process uses the configuration file specified by the -config parameter to import JSON data into the target server and display it in the Ultipa graph structure.

./ultipa-importer --config import.sample.json.yml