This project is read-only.
Lazily Enumerate CSV File

Category:
Input/Output
Description: Build an IEnumerable for on-demand reading of a CSV file using .NET I/O utilities and abstractions
Code:
let EnumerateCSVFile1() = 

// Write a test file
System.IO.File.WriteAllLines(@"test.csv", [| "Desmond, Barrow, Market Place, 2";
"Molly, Singer, Band, 12" |]);

/// This function builds an IEnumerable object that enumerates the CSV-split
/// lines of the given file on-demand
let CSVFileEnumerator(fileName) =

// The function is implemented using a sequence expression
seq { use sr = System.IO.File.OpenText(fileName)
while not sr.EndOfStream do
let line = sr.ReadLine()
let words = line.Split [|',';' ';'\t'|]
yield words }

// Now test this out on our test file, iterating the entire file
let test = CSVFileEnumerator(@"test.csv")
printfn "-------Enumeration 1------";
test |> Seq.iter (string >> printfn "line %s");
// Now do it again, this time determining the numer of entries on each line.
// Note how the file is read from the start again, since each enumeration is
// independent.
printfn "-------Enumeration 2------";
test |> Seq.iter (Array.length >> printfn "line has %d entries");
// Now do it again, this time determining the numer of entries on each line.
// Note how the file is read from the start again, since each enumeration is
// independent.
printfn "-------Enumeration 3------";
test |> Seq.iter (Array.map (fun s -> s.Length) >> string >> printfn "lengths of entries: %s")

Execution Result:
-------Enumeration 1------
line System.String[]
line System.String[]
-------Enumeration 2------
line has 8 entries
line has 7 entries
-------Enumeration 3------
lengths of entries: System.Int32[]
lengths of entries: System.Int32[]

Last edited Sep 14, 2011 at 4:08 AM by ttliu2000, version 1

Comments

gjvdkamp Sep 10, 2012 at 10:11 PM 
The part 'seq{ .. }' can be replaced with
File.ReadLines filename |> Seq.map (fun s -> s.Split '\t' )
to save on typing

RainerSchuster Oct 17, 2011 at 11:19 PM 
I've created a more advanced Reader and simple writer extensions to seq on fssnip.org inspired by Dons Expert F# 2.0 Samples. http://fssnip.net/tags/CSV