This manual covers the usage of Ultipa Exporter (Go Version), a command-line-based lightweight tool for fast export of multiple metadata files from the Ultipa Graph database to local in one command.
Change Log (V4.1 to V4.2)
- Supports text type
- Adds
MaxPacketSize
to adjust the maximum bytes of each packet the SDK processes - Adds
Timeout
to set grpc timeout limit - Adds
timezone
to set for timestamp - Checks whether graphset status is MOUNTED before export
Prerequisites
- configuration file
- a command line terminal that is compatible with your operating system:
- Linux or MacOS: bash, zsh, tcsh
- Windows: PowerShell
- a version of Ultipa Exporter compatible with your operating system
Background Knowledge - System Properties
System properties of node:
_id
: ID of node, a string of maximum 128 bytes_uuid
: ID of node, an uint64
System properties of edge:
_uuid
: ID of edge, an uint64_from
: the_id
of start-node (FROM) of edge_to
: the_id
of end-node (TO) of edge_from_uuid
: the_uuid
of start-node (FROM) of edge_to_uuid
: the_uuid
of end-node (TO) of edge
System properties will be exported by default and do not need to be specified in the configuration file.
Data File
- File name:
<schema>.node.<file_type>
and<schema>.edge.<file_type>
, i.e., 'default.node.csv' - Each file: nodes or edges that belong to a specific schema
- Each row (except headers): a node or an edge
- Each column: a property
- File type:
csv
- File delimiter:
,
- File headers (column names) format:
<property_name>:<property_type>
, set parameterwriteHeader
to generate headers or not
YML: server
server:
host: "192.168.35.151:60024" # for cluster, separate multiple server nodes with comma ','
username: employee533
password: joaedSSGsdf
crt: "" # The directory of the SSL certificate when both servers are in SSL mode
graphset: test_graph # The graphset name, or use graphset 'default' by default
YML: nodeConfig | edgeConfig
nodeConfig:
- schema: student # The schema to export, mandatory
# export all properties of this schema when not specifying `properties`
- schema: course
properties:
- name: title # Declare custom properties to be exported
- name: credit
# System properties _id、_uuid will be exported by default and do not need to be declared
edgeConfig:
- schema: "*" # Star * denotes all properties of all schemas
YML: settings
settings:
writeHeader: false # Whether to write headers into the file, or do by default
outPath: ./temp # The path of the exported files, or write to './export/' by default
Other parameters:
Parameter | Specification | Default Value | Description |
---|---|---|---|
MaxPacketSize | int | 41943040 (40M) | The maximum bytes of each packet the GO SDK processes |
Timeout | int | 1000 | The grpc timeout limit when exporting huge amount of data, unit in second; set to <0 for no limit, set to 0 for 1000s |
timezone | string | (local timezone) | The timezone of timestamp values, e.g. +00:80, Asia/Shanghai etc. |
Command Line
- Show help
./ultipa-exporter --help
- Download configuration sample file
./ultipa-exporter --sample
- Execute export operation, the config file out.yml is in the current directory
./ultipa-exporter --config ./out.yml
All parameters:
Command |
Description |
---|---|
--help | show help information |
--config <FILE_PATH_NAME> | define the configuration file and execute import operation |
--sample | true: generate a sample config file; false: do not generate sample config file |
--host <IP:PORT> | overwrite the parameter host in the config file |
--graph <GRAPH_NAME> | overwrite the parameter graphset in the config file |
--username <USERNAME> | overwrite the parameter username in the config file |
--password <PASSWORD> | overwrite the parameter password in the config file |
--maxPacketSize <MAXPACKETSIZE> | overwrite the parameter MaxPacketSize in the config file |
--logAppend | true: append multiple error info into one log file; false: generate a log file for each error info |
--progressLog <boolean> | (for Ultipa Manager) true: generate progress log; false do not generate progress log |
--version | true: show Ultipa Exporter version; false: do not show Ultipa Exporter version |
FAQ
Q: I got such error 'rpc error: code = ResourceExhausted desc = Sent message larger than max (31324123 vs. 4194304)', what does it mean and how to solve it?
A: This message means when exporting a data batch, the packet size which is 31324123 bytes exceeds the limit of 4194304 bytes. The possible reasons are too many properties imported at a time, or excessive property volume (long texts stored in text type), as a result of which the data volume of a data batch exceeds the default server config of max_rpc_msgsize (4M) and/or the MaxPacketSize of Go SDK (40M).
Solution: raise the setting of MaxPacketSize
in the config file, and/or max_rpc_msgsize
in the server config (the latter requires a server re-boot). Ultipa Exporter automatically decides the batch size which cannot be set by users.
Q: What timezone is used when exporting time values?
A: Value of datetime has no timezone information, value of timestamp will be exported according to the parameter timezone
setting, or in local timezone if timezone
is not set.