Generic Separated File Loader

The same compiler is used for comma, tab and vertical pipe separated files. This loader compiles the contents of the file into a list of lists of fields. The csv, tab and vsv files all share the same basic loader.

File Format

The most basic unit of this format is the field. Fields are separated by ",", "|" or a tab. The respective field separator can appear in a field if it is quoted. A quoted field is surrounded by the double quote character ". All the fields on a line are stored as a list and these lists are stored in another list representing the whole file. e.g. a example comma separated file:

R1F1, R1F2, R1F3, "R1F4", R1F5
field 1,"field 2,", "field 3" ,field 4 , field 5
item
3, "item
4"
        

The above would return a list of lists of strings. There are four records, which contain varying numbers of fields. In general one record would be one line, however note the last item which uses quotes to include a line break. Some of those fields are quoted, note the effect of line breaks in fields and the use of quotes to change that. Below is the data structure millscript would hold.

[
  [R1F1, R1F2, R1F3, R1F4, R1F5],
  [field 1, field 2, field 3, field 4, field 5],
  [item],
  [3, item
4]
]
        

In the above example, the only difference for a Tab or | separated file is that the "," characters are replaced with either a tab or "|".

Use

  1. Create an csv file in your inventory, containing all the required records, as detailed in the file format. e.g Lets assume you create a file called hotels.csv.
  2. To use the connection within a script, we just have to refer to the name of the jdbcconn file, e.g.
    hotels
                
    and this will load the connection for us. This can then be used just like any other list.

Notes