Deepgreen DB

Xdrive FS Plugin

FS plugin lets Deepgreen read/write from/to local filesystem on the servers running Xdrive. It currently support two file formats: CSV and SPQ.

Mountpoint

Suppose we store data in the directory /dw/data, the following mountpoints would allow access to the data:

[[xdrive.mount]]
name = "datacsv"                # mountpoint name
argv = ["xdr_fs/xdr_fs",               # plugin
     "csv",                     # file format (csv or spq)
     "/dw/data",                # root data dir
     ]

DDL

When constructing an external table DDL that refers to a FS Plugin, specify the LOCATION clause like this for Reads:

LOCATION(`xdrive://XDRIVE-HOST-PORT/MOUNTPOINT/path-to-files-with-wildcard.{csv|spq}`)
FORMAT '{CSV|SPQ}'

And for Writes:

LOCATION(`xdrive://XDRIVE-HOST-PORT/MOUNTPOINT/path-to-files-with-#UUID#.{csv|spq}`)
FORMAT '{CSV|SPQ}'

For example, To read/write the nation csv table located in /dw/data/public/nation directory, we create external table DDLs like this:

DROP EXTERNAL TABLE IF EXISTS nation; 
CREATE EXTERNAL TABLE nation (
    n_nationkey integer,
    n_name varchar,
    n_regionkey integer,
    n_comment varchar)
LOCATION ('xdrive://XDRIVE-HOST-PORT/datacsv/public/nation/nation*.csv')
FORMAT 'CSV';

DROP EXTERNAL TABLE IF EXISTS nation_writer;
CREATE WRITABLE EXTERNAL TABLE nation_writer (
    n_nationkey integer,
    n_name varchar,
    n_regionkey integer,
    n_comment varchar)
LOCATION ('xdrive://XDRIVE-HOST-PORT/datacsv/public/nation/nation#UUID#.csv')
FORMAT 'CSV';

Note that to read / write SPQ and CSV files, you would need separate mountpoints.