Add a header to begin generating the table of contents
This page refers to APL tools for reading and writing data to and from files, databases, and the internet. If you are already familiar with Python, R or .NET then you can use one of the external language bridges to bring data into APL from files via one of these languages. However, it can be simpler and faster in many cases to use one of the following tools. The tools and interfaces catalogue lists more tools and links to their documentation.
Add a header to begin generating the table of contents
Import while Developing
The experimental ]Get user command can be used in the interactive IDE to obtain code and data from the internet or local file system in various formats. For example:
- APL code from files, folders and online repositories like GitHub
- Workspaces and text source shipped with the interpreter, for example dfns and HttpCommand
- Text data including plain text, CSV, XML and JSON
]Get is a development tool intended as a one-stop utility for quickly bringing resources into the workspace while programming. Do not use it at run time, as exact results can vary. Instead, use precisely documented features like ⎕JSON, ⎕CSV, ⎕XML, and ⎕FIX in combination with loading tools like ⎕NGET, HttpCommand, ⎕SE.Link.Import, etc.
Enter ]Get -? into the interactive session to see more information.
From the Internet
HttpCommand is a utility for making requests to interact with web services.
Load HttpCommand into the active workspace.
]Get HttpCommand
#.HttpCommand (HttpCommand.Get 'https://catfact.ninja/fact').Data
{"fact":"Cats have about 130,000 hairs per square inch (20,155 hairs per square centimeter).","length":83} (HttpCommand.GetJSON 'GET' 'https://catfact.ninja/fact').Data.fact
There are approximately 60,000 hairs per square inch on the back of a cat and about 120,000 per square inch on its underside. r←HttpCommand.Get 'https://catfact.ninja/fact'
r.(HttpStatus HttpMessage)
┌───┬──┐
│200│OK│
└───┴──┘Text Files
⎕NGET and ⎕NPUT can be used to read and write text files.
(⊂words)⎕NPUT'data/words.txt' ⍝ Write words to a unicode text file
(content encoding newline)←⎕NGET'data/words.txt' ⍝ Read words from a unicode text file
words←⊃⎕NGET'data/words.txt' 1 ⍝ Split words on each new line CSV
The Comma Separator Values system function
⎕CSV can read tabular data from .csv files as APL matrices. Here are some features of ⎕CSV: -
Read data from and write data to files directly.
data ← ⎕CSV '/path/to/file.csv' ⍝ Read from file.csv data ⎕CSV '/path/to/file.csv' ⍝ Write to file.csv -
Separate the header (first row) from the rest of the data.
(data header) ← ⎕CSV '/path/to/file.csv' ⍬ ⍬ 1 -
Import specific columns as numbers or characters, depending on the options provided.
The 4 in this example indicates to convert numeric values if possible, and otherwise keep the value as text.numeric_if_possible ← ⎕CSV '/path/to/file.csv' ⍬ 4 -
Use a separator other than commas, using the “Separator” variant option, for example using tabs (⎕UCS 9) for Tab Separated Values (.tsv).
tsv ← ⎕CSV⎕OPT'Separator' (⎕UCS 9)⊢'/path/to/file.tsv'
JSON
JavaScript Object Notation can be translated to and from APL.
-
Lists can be represented as APL vectors.
1⎕JSON (1 2 3)'ABCD' [[1,2,3],"ABCD"] -
Objects can be represented as APL namespaces.
0⎕JSON '{"name":"David", "age": 42}' #.[JSON object] -
Both can be represented as a matrix of depth, name, value and type columns somewhat similar to that used by ⎕XML.
0 (⎕JSON ⎕OPT'Format' 'M')'[{"name":"David", "age": 42}, {"name": "Sandra", "age": 42}]' ┌─┬────┬──────┬─┐ │0│ │ │2│ ├─┼────┼──────┼─┤ │1│ │ │1│ ├─┼────┼──────┼─┤ │2│name│David │4│ ├─┼────┼──────┼─┤ │2│age │42 │3│ ├─┼────┼──────┼─┤ │1│ │ │1│ ├─┼────┼──────┼─┤ │2│name│Sandra│4│ ├─┼────┼──────┼─┤ │2│age │42 │3│ └─┴────┴──────┴─┘
XML
⎕XML converts between XML character vectors and a nested matrices of node depth, tag name, value, attribute key/value pairs and markup description columns.
⎕XML'<name born="1920">Ken</name><name born="1925">Jean</name>'
┌─┬────┬────┬───────────┬─┐
│0│name│Ken │┌────┬────┐│5│
│ │ │ ││born│1920││ │
│ │ │ │└────┴────┘│ │
├─┼────┼────┼───────────┼─┤
│0│name│Jean│┌────┬────┐│5│
│ │ │ ││born│1925││ │
│ │ │ │└────┴────┘│ │
└─┴────┴────┴───────────┴─┘Regular Expressions
Regular expressions are a powerful tool for text processing and expressions for many tasks are easily searched online.
Convert what you say into what your dog Rex hears.
Rex ← 'rex' '\w'⎕R'\0' '*'⍠1
Rex 'Rex, I told you not to do that! Now, Rex, go and wait outside'
Rex, * **** *** *** ** ** ****! ***, Rex, ** *** **** *******
In many cases, transformations can be much faster in APL. See how to trim spaces from text with APLCart.
Dates and Times
Convert between every computer datetime format (that we could think of) with the ⎕DT system function.
now ← 1613552859 ⍝ Unix Timestamp
⍝ 1200⌶ is experimental
'Dddd Doo Mmmm YYYY hh:mm' (1200⌶) 20 1 ⎕DT now
┌──────────────────────────────────┐
│Wednesday 17th February 2021 09:07│
└──────────────────────────────────┘1200⌶ function converts between numeric date numbers and human-readable datetimes. Binary Files and Other Arbitrary File Types
The term “Native Files” refers to any type of file on a hard disk. Some system functions beginning ⎕N can be used to read and write files of arbitrary type and format.
tn←'words.txt'⎕NTIE 0 ⍝ Tie the file, locking it from use by other processes
⎕←10↑⎕NREAD tn 80 ¯1 ⍝ Read the data as Unicode text
A
A's
AA's
⎕NUNTIE tn ⍝ Untie the fileMemory-mapped Files
⎕MAP allows you to treat a file on disk as if it were a variable in the workspace. This is useful if you are working with data that cannot fit inside the available workspace memory. One approach might be to read the data in chunks and process one chunk at a time (for example, see the “Records” variant option for ⎕CSV). Another approach is to use ⎕MAP.
text ← 80 ¯1 ⎕MAP '/path/to/file.txt'APL Component Files
⎕MAP allows you to treat a file on disk as if it were a variable in the workspace. This is useful if you are working with data that cannot fit inside the available workspace memory. One approach might be to read the data in chunks and process one chunk at a time (for example, see the “Records” variant option for ⎕CSV). Another approach is to use ⎕MAP.
text ← 80 ¯1 ⎕MAP '/path/to/file.txt'SQL Databases
SQAPL ships with Dyalog and can be used out-of-the-box provided that a database is installed and a corresponding ODBC data source has been set up.
'SQA'⎕CY'sqapl'
SQA.Connect cid odbc_datasource_name sql_password sql_user
SQA.Do cid 'USE my_database'
SQA.Do cid 'SELECT * FROM my_table'
