About Site Map Submit Contact Us Log in | Create an account
Create an account Log In
Average Rating
User Rating:
Visitors Rating:
My rating:

Write review
See full specifications

scripts default iconError-tolerant parsing of newline-delimited data (Scripts) Publisher's description

Adaptive parsing of newline-separated data. Handles bad lines WITHOUT reading line-by-line.

Adaptive parsing of newline-separated data. Handles bad lines WITHOUT reading line-by-line.
Generally, when parsing newline-delimited data, it is fastest to parse
all the lines in a single step, using "vectorized" functions like fscanf, rather than looping over the data line-by-line.

A single corrupt line in your data, however, can cause your parsing
function to fail. When this happens, the easiest workaround is to resort
to line-by-line parsing, but this can be extremely slow.

adaptive_parse.m provides a means to apply efficient parsing functions
to newline-delimited data containing corrupt lines. The user supplies
a parsing function, customised for his/her specific data file format,
which is passed to adaptive_parse as a function handle. If no corrupt
data lines are encountered, adaptive_parse calls the parsing function only once, returning the parsed data.

If the parsing function throws an error due to encountering a corrupt line, adaptive parse "rewinds" back to its prior position in the data and retries the parsing with half the number of lines. This is done iteratively until the bad line is read in singly and rejected. After this happens, adaptive_parse resumes parsing, rapidly ramping up the number of lines being parsed for greater speed.

Parsing a large number of lines at a time speeds up execution, but may
also result in Matlab exceeding its memory capacity. The user can specify a maximum number of lines to be parsed at a time to avoid this, or s/he can allow adaptive_parse to handle out-of-memory errors automatically. adaptive_parse.m responds to an out-of-memory error in
a similar way as it does to corrupt data lines: it iteratively halves
the number of lines being parsed until the error no longer occurs (a caveat: this mechanism works under Linux, but in Windows XP, Matlab seems to be unable to recover from the out-of-memory condition, and it freezes up).

The behaviour and efficiency of adaptive_parse.m can be modified through the use of optional initialisation, appender and cleanup functions, also provided by the user. Demonstration functions of these types have been included with this distribution of adaptive_parse.m, along with a demonstration parsing function and file of data that has been deliberately been corrupted to illustrate adaptive_parse's functionality.

System Requirements:

No special requirements.
Program Release Status: New Release
Program Install Support: Install and Uninstall

Error-tolerant parsing of newline-delimited data (Scripts) Tags:

Click on a tag to find related softwares

Is Error-tolerant parsing of newline-delimited data (Scripts) your software?

Manage your software

Most Popular

scripts default icon ASK, OOK, FSK, BPSK, QPSK, 8PSK modulation 1.1
ASK, OOK, FSK, BPSK, QPSK, 8PSK modulation contain several functions for digital modulation simulation
scripts default icon Simulink Communication Labs 1.1
Simulink Communication Labs allows you to learn communication systems in greater depth.
scripts default icon M-QAM modulation and demodulation 1.1
M-QAM modulation and demodulation is the QAM modulation and demodulation tech.
scripts default icon LZW Compression/Decompression 1.1
LZW Compression/Decompression - Updated LZW compressor and decompressor with reasonable performance
scripts default icon InSPIRE utility to plot a 2D displacement field (Scripts) 1.0
This program plots the deformation field (displace vectors) contained in vector.txt.