flipper resurfaces

This is the page for the flipper utility.  As it stands now, excluding bug fixes, this major revision of June 30 of July 2, 2011 will remain pretty much the way it is right now.  Changes across versions are listed at the bottom of this page.

 flipper is a java command line program for transposing explicit.txt and demographics.txt files (from either research or demo sites).  The following is the feature list:

1. Takes either demographics or explicit file as input and produced flipped text file. These inputs are assumed to be in the standard format (tab delimited, variables in standard order).
2. Output is sorted in order of session_id  or user_id. Within each row, the variables are sorted alphanumerically.
3. Variables can be filtered out.  So if you want to exclude all variables that dont appear more than 20 times that is possible. By default, this threshold is set at 10.
4. Variables can be excluded by pattern.  For example,  all variables that contain the string "rt" in them can be excluded.
5. Variables can be included by pattern.  For example,  only variables that contain the string "sdo" can be marked for flipping.
6. Multiple variable patterns can be marked for inclusion and exclusion by using comma as a separator.
7. The file logs.txt contains a fairly detailed log of the processing.
8. The data file can be annotated and have blank or other weird lines. flipper tries to ignore these lines.
9. For very large datasets, you'll probably need to allocate lots of memory (64 bit java can do this) as it has not been optimized for memory usage.  I have run it on datasets with over 30,000 sessions.

To use this utility, follow these steps:

1. Download the flipper.zip file from this page.
2. Rename the file  to flipper.class before using it.  As this system did not allow flipper.class to be uploaded, I am uploading it as a zip file. Do not try to unzip this file, because it is not a zip file. You could place it in a folder that is in the CLASSPATH environment variable; this makes it available for use on any folder in the system (without copying it there).
3. Copy the attached flipper.class file to the folder where you are keeping your data. 
4. Open the program or application on your computer that allows you to type in command lines.  On a Mac, this is called "Terminal", and you can find it under Applications/Utilities.
5. Set your working directory (the folder that the command line program will automatically use).  On a Mac, you can do this by typing "cd" and then the file path.  For example, if your data is on your desktop, you would type "cd /Users/mst3k/Desktop", replacing "mst3k" with your computer's username.
6. To flip an explicit file, type "java flipper explicit.txt expflipped.txt" into the command line and press Enter.  For largish data, use "java -Xmx1024m flipper explicit.txt expflipped.txt" instead.  This allocates more memory than the default and will work on most computers.  On 64-bit systems, it is possible to run flipper on very large data files using special command-line options, although currently it is not optimized for such data sets.  If you have an explicit file, for example, you could try   Other examples for pattern matching are provided when you start the program (even without any parameters).  
7. You can check the logs file for errors or any other anomalies that it encountered. The frequencies of selected variables are also listed in the log file. 

If major bugs are found/reported, I'll update this page with any fixes.

Versions

1. June 30, 2011.  First version posted here.

2. July 2, second version.
       Ignores missing data during processing (but writes them out as periods).
       Accepts multiple include and exclude patterns.
       Progress indicator every 5% of the data.

3. July 15, bug fix for errors in small files less than 2000 lines.

4. October 24th, edits for clarification