From jaksonaquino at yahoo.com.br Wed Jan 11 17:25:57 2006 From: jaksonaquino at yahoo.com.br (Jakson A. Aquino) Date: Wed, 11 Jan 2006 14:25:57 -0200 Subject: The next stable version Message-ID: <20060111162557.GA7913@localhost.localdomain> Hello! I wrote a script that compares the statistical results of the current cvs version of statist with the results of the version that was in the cvs in the first half of 2005. I have found two differences. One was a bug in the cvs version that was fixed yesterday (u-test). Now, the only difference between the current version of statist at the cvs and the old stable version is one bug in the old version at "Partial linear correlation" when we run this function with five variables. Thanks to valgrind this bug was fixed in statist-1.3.1. The script might not catch all differences in the statistical results between the two versions. Thus, everyone is invited to do new tests. Anyway it seems that the current cvs version is, regarding statistics, at least as accurate as the old version. In addition, the current version doesn't consider that user errors are fatal errors: it crashes less, and quits abruptly less. Thus it might be considered the new stable version. The script can be found here: http://intevation.de/cgi-bin/viewcvs-statist.cgi/statist/tools/run_comparison.sh Best regards, Jakson -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://www.intevation.de/pipermail/statist-list/attachments/20060111/dca7a915/attachment.bin From jaksonaquino at yahoo.com.br Fri Jan 13 16:27:54 2006 From: jaksonaquino at yahoo.com.br (Jakson A. Aquino) Date: Fri, 13 Jan 2006 13:27:54 -0200 Subject: Statist 1.4.0 - Stable version released! Message-ID: <20060113152754.GA4931@localhost.localdomain> Lightweight statist(ical) application release! ----------------------------------------------------------------- Statist 1.4.0 is available! This is the new stable version The statist community likes your feedback. You can send your message to the mailing list available from the homepage or, if you prefer, you can write directly to Jakson Aquino, the statist maintainer. WHAT IS STATIST? =============== Statist is a small and portable statistics program written in C. It is terminal-based, but can use Gnuplot for plotting purposes. Scripting is possible and Statist handles big datasets reasonably well on small machines. It is also easy to use due to its interactive menu, being a very handy tool also for those who do not use statistics every day. Statist is Free Software under GNU GPL v>=2. THE DOCUMENTATION ================= The statist user manual is available in both English and Portuguese. The German documentation is partially outdated, but it contains information about the statistical functions and the way the program works that are not covered by the new user manual. REQUIREMENTS ============ Statist is very portable. It can be compiled in practically any system with any C compiler. However, and the system must have libintl and libiconv to compile the program with support for languages other than English. If gnuplot is installed, statist can create graphics. Some GNU core utils -- gawk and sort -- are useful additions to prepare databases for use with Statist. DISTRIBUTION ============ Statist source code is available at its homepage: http://wald.intevation.org/projects/statist/ BUGS FIXED IN VERSION 1.4.0 =========================== * Fixed bug in u-test. Rows with missing values were being deleted. This bug was introduced in the development version 1.3.0 (August 2005). * Fixed segmentation fault when the first column is marked as a column to be ignored (prefix "$" in the column name). * Fixed incompatible pointer type when compiling in amd64. * Sandro Tosi revised all strings from the Italian catalog, and translated the new ones. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://www.intevation.de/pipermail/statist-list/attachments/20060113/423b0b5a/attachment.bin From jalvesaq at gmail.com Tue Jul 4 23:10:46 2006 From: jalvesaq at gmail.com (Jakson A. Aquino) Date: Tue, 4 Jul 2006 18:10:46 -0300 Subject: utf-8 and gnuplot Message-ID: <20060704211046.GA28844@localhost.localdomain> Hello! People using statist with their locale charset set to UTF-8 might have problems with gnuplot graphics if they contain labels or titles with accented letters. Some months ago, I added a work around to statist plot.c: all strings were converted into a single byte charset, like ISO-8859-1. However, since April, 2006, the cvs version of gnuplot has an improved handling of UTF-8 strings and I think that my work around is no longer necessary. I didn't remove the changes that I've made yet only because it's somewhat difficult to discover the correct font specification for gnuplot. If the user has no intention of mixing charsets like Latin and Greek in the graphics, it's easier to set the variable gnuplot_charset in the ~/.statistrc. I put a discussion about this issue here: http://statist.wald.intevation.org/utf8.html Best regards, Jakson -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://www.intevation.de/pipermail/statist-list/attachments/20060704/655c9c40/attachment.bin From bernhard at intevation.de Wed Jul 5 15:30:56 2006 From: bernhard at intevation.de (Bernhard Reiter) Date: Wed, 5 Jul 2006 15:30:56 +0200 Subject: utf-8 and gnuplot In-Reply-To: <20060704211046.GA28844@localhost.localdomain> References: <20060704211046.GA28844@localhost.localdomain> Message-ID: <200607051530.57343.bernhard@intevation.de> Hi Jakson, Am Dienstag, 4. Juli 2006 23:10 schrieb Jakson A. Aquino: > I put a discussion about > this issue here: > > http://statist.wald.intevation.org/utf8.html sound all like a cool improvement! Best, Bernhard From jalvesaq at gmail.com Thu Sep 7 13:15:25 2006 From: jalvesaq at gmail.com (Jakson A. Aquino) Date: Thu, 7 Sep 2006 08:15:25 -0300 Subject: reading data from csv files Message-ID: <20060907111525.GA21315@localhost.localdomain> Hello All, I've made some changes in the algorithm used by statist to open data files. All files that were opened should continue to be read as before, but now statist is capable of reading .csv files as created by spreadsheet programs. Statist should open most files without problems, automatically detecting if the file has a header with variable names and correctly parsing the lines with data. To achieve this goal, among other changes, I added double quotes and commas to the ignore[] string (data.c), used to distinguish between data and field separators. I added the following options to let users fine tune statist behavior if it fails to open a file correctly: --header : the file has column names in the first line --noheader : the file does not have column names --sep : field separator character --dec : decimal delimiter character (default: '.') --na-string : indicator of missing values (default: "M") In the scripts with tests that are attached to this e-mail, it was possible to open files with a wide range of formatting styles, and it was necessary to use command line options only when the field separator was an empty space and the decimal delimiter was a comma, as can be seen in the table below: Command line options to open a data file with statist according to how the data is formatted ============================================================================ | dec = . | dec = , |----------------------|--------------------- | sep = , | sep = ' ' | sep = , | sep = ' ' -------------------------------|----------|-----------|---------|----------- header | quoted data | | | | (statist style)|---------------|----------|-----------|---------|----------- ( #% ) | not quoted | | | | --dec "," ---------------|---------------|----------|-----------|---------|----------- | quoted data | | | | header |---------------|----------|-----------|---------|----------- | not quoted | | | | --dec "," ---------------|---------------|----------|-----------|---------|----------- | quoted data | | | | no header |---------------|----------|-----------|---------|----------- | not quoted | | | | --dec "," ============================================================================ Note: the combination of (data not quoted & dec = "," & sep = ",") is possible only for integer values. I also updated the documentation to reflect the changes, and if no problem is detected in the new algorithm, it will be part of the next release of statist. For now, the changes are available in the cvs: cvs -z3 -d:pserver:anonymous at cvs.intevation.de:/home/statist/jail/statistrepository co statist One disadvantage of these new features is that statist is taking about 30% more time to load a file. All commentaries and suggestions are welcome! Best regards, Jakson -------------- next part -------------- A non-text attachment was scrubbed... Name: test_statist.tar.gz Type: application/octet-stream Size: 2797 bytes Desc: not available Url : http://www.intevation.de/pipermail/statist-list/attachments/20060907/d39da57b/test_statist.tar.gz -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://www.intevation.de/pipermail/statist-list/attachments/20060907/d39da57b/attachment.bin From beyer at imb-jena.de Thu Sep 7 18:47:07 2006 From: beyer at imb-jena.de (Andreas Beyer) Date: Thu, 07 Sep 2006 09:47:07 -0700 Subject: reading data from csv files In-Reply-To: <20060907111525.GA21315@localhost.localdomain> References: <20060907111525.GA21315@localhost.localdomain> Message-ID: <45004D0B.2040707@imb-jena.de> Hi Jakson, Your extensions are certainly valuable! Particularly I like automatic detection of the input format. Fine-tuning via command line options is good, but eventually you may want to allow for defining the details when opening the file (in interactive mode). This would allow to define details for every file separately. For instance there could be a sub-menu DATA MANAGEMENT/FILE FORMAT OPTIONS Andreas Jakson A. Aquino wrote: > > I added the following options to let users fine tune statist behavior if it > fails to open a file correctly: > > --header : the file has column names in the first line > --noheader : the file does not have column names > --sep : field separator character > --dec : decimal delimiter character (default: '.') > --na-string : indicator of missing values (default: "M") > > From jalvesaq at gmail.com Thu Sep 7 21:46:44 2006 From: jalvesaq at gmail.com (Jakson A. Aquino) Date: Thu, 7 Sep 2006 16:46:44 -0300 Subject: reading data from csv files In-Reply-To: <45004D0B.2040707@imb-jena.de> References: <20060907111525.GA21315@localhost.localdomain> <45004D0B.2040707@imb-jena.de> Message-ID: <20060907194644.GA18280@localhost.localdomain> Hi Andreas, On Thu, Sep 07, 2006 at 09:47:07AM -0700, Andreas Beyer wrote: > Fine-tuning via command line options is good, but eventually you may > want to allow for defining the details when opening the file (in > interactive mode). This isn't difficult to implement. We have just to define the better way of doing it. > This would allow to define details for every file separately. For > instance there could be a sub-menu DATA MANAGEMENT/FILE FORMAT > OPTIONS I can add an option to statistrc defining whether statist will try to auto detect the file format or ask the user. In auto detect mode (and not in silent mode), if some error happens (different number of columns for different lines), the user would be asked to set the options. Before asking the questions, statist could show the first few lines (truncated after about 40 characters), just as it shows the content of the current directory when asking for a file name. The sub-menu DATA MANAGEMENT/FILE FORMAT is useful too. In auto detect mode, statist will read: 1,1 2,2 1 1 2 2 1,1 2,2 as 1 1 2 2 1,1 2,2 1 1 2 2 Because statist would find the same number of columns in all lines, there would be no error message. Only the presence of column names would avoid that statist wrongly read this file. The sub-menu item would make statist re-read the file with the correct options. Would this statist behavior be OK in your opinion? Or were you think in something different? Jakson -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://www.intevation.de/pipermail/statist-list/attachments/20060907/831bbfab/attachment.bin From jalvesaq at gmail.com Wed Nov 15 02:14:30 2006 From: jalvesaq at gmail.com (Jakson A. Aquino) Date: Tue, 14 Nov 2006 23:14:30 -0200 Subject: untranslated messages Message-ID: <20061115011430.GA4842@localhost.localdomain> Hello All! The cvs version of statist fixes some bugs regarding UTF-8 locales and can open more file types than before. Because currently I'm not planning to add new features, I think it's a good time to release a new version of statist. As usual, there are some untranslated messages and I would be grateful if we have volunteers to translate them (about 30 messages in German and Italian, and about 100 in Spanish) Thank you! Jakson -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://www.intevation.de/pipermail/statist-list/attachments/20061114/7dc02bfb/attachment.bin From jalvesaq at gmail.com Sun Nov 26 22:35:35 2006 From: jalvesaq at gmail.com (Jakson A. Aquino) Date: Sun, 26 Nov 2006 19:35:35 -0200 Subject: untranslated messages In-Reply-To: <20061115011430.GA4842@localhost.localdomain> References: <20061115011430.GA4842@localhost.localdomain> Message-ID: <20061126213535.GA5427@localhost.localdomain> Hello All! Thanks to Michael Gebhardt, the German translation is complete. Now, I plan to release statist 1.4.1 at the next weekend. Best regards, Jakson On Tue, Nov 14, 2006 at 11:14:30PM -0200, Jakson A. Aquino wrote: > The cvs version of statist fixes some bugs regarding UTF-8 locales > and can open more file types than before. Because currently I'm > not planning to add new features, I think it's a good time to > release a new version of statist. As usual, there are some > untranslated messages and I would be grateful if we have > volunteers to translate them (about 30 messages in German and > Italian, and about 100 in Spanish) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://www.intevation.de/pipermail/statist-list/attachments/20061126/8f52d2a8/attachment.bin From bernhard at intevation.de Thu Nov 30 19:12:40 2006 From: bernhard at intevation.de (Bernhard Reiter) Date: Thu, 30 Nov 2006 19:12:40 +0100 Subject: untranslated messages In-Reply-To: <20061126213535.GA5427@localhost.localdomain> References: <20061115011430.GA4842@localhost.localdomain> <20061126213535.GA5427@localhost.localdomain> Message-ID: <200611301912.41064.bernhard@intevation.de> On Sunday 26 November 2006 22:35, Jakson A. Aquino wrote: > Thanks to Michael Gebhardt, the German translation is complete. > > Now, I plan to release statist 1.4.1 at the next weekend. Cool. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/statist-list/attachments/20061130/828fc8e6/attachment.bin From jalvesaq at gmail.com Sat Dec 2 21:55:02 2006 From: jalvesaq at gmail.com (Jakson A. Aquino) Date: Sat, 2 Dec 2006 18:55:02 -0200 Subject: Statist 1.4.1 Released Message-ID: <20061202205502.GA2722@localhost.localdomain> Lightweight statist(ical) application release! ----------------------------------------------------------------- Statist 1.4.1 is available! The statist community likes your feedback. You can send your message to the mailing list available from the homepage. WHAT IS STATIST? ================ Statist is a small and portable statistics program written in C. It is terminal-based, but can use Gnuplot for plotting purposes. Scripting is possible and Statist handles big datasets reasonably well on small machines. It is also easy to use due to its interactive menu, being a very handy tool also for those who do not use statistics every day. Statist is Free Software under GNU GPL v>=2. THE DOCUMENTATION ================= The statist user manual is available in both English and Portuguese. The German documentation is partially outdated, but it contains information about the statistical functions and the way the program works that are not covered by the new user manual. REQUIREMENTS ============ Statist is very portable. It can be compiled in practically any system with any C compiler. However, and the system must have libintl and libiconv to compile the program with support for languages other than English. If gnuplot is installed, statist can create graphics. Some GNU core utils -- gawk and sort -- are useful additions to prepare databases for use with Statist. DISTRIBUTION ============ Statist source code is available at its homepage: http://statist.wald.intevation.org/ NEW FEATURES IN VERSION 1.4.1 ============================= * Statist now can open csv files. It will try to discover whether the file has a header with column names, what is the field separator, and what is the decimal separator. If necessary, these parameters might be set using command line options. * Gnuplot graphics now can have strings with letters of different alphabets if the environment charset is UTF-8 and the user chooses UTF-8 as the "gnuplot_charset" in the statistrc file. * Added menu item "Save last gnuplot graphic as png". * Added statistrc options "gnuplot_default_term" and "gnuplot_png_font". * Do not add the string "#%" to files created with --xcols if either the command line option --header was used or the statistrc option autodetect_header is active. BUGS FIXED IN VERSION 1.4.1 =========================== * No longer crashing if gnuplot isn't installed and the user tries to run a graphic. * No longer crashing on menu item 4.1 if there is a "nan" value in the datafile. Now, the string "nan" is read as missing value. * Fixed wrong alignment of columns in "Frequency table", "Compare means", and "Show data of columns" when the locale charset was UTF-8, and there is non-ascii characters. * The min and max values chosen by the user in menu item 4.1 now are used used even if the number of classes wasn't chosen too. * Partial correlation with five variables now prints some correlations that were missing in the output. * The function extract_cols() now correctly extract non-numeric values from fixed width column files, but outputs a warning if such a value is found. * "Yes/No" questions might be answered in the language that statist is running. * The alert character (^G) no longer is written to statist.log. * Fixed crash in out_r() on AMD64 when the option --log was active. * Removes the GPL_DAT file when the user sends to gnuplot the "quit" command. * Michael Gebhardt translated the new strings into German. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://www.intevation.de/pipermail/statist-list/attachments/20061202/f45b6cc9/attachment.bin From bernhard at intevation.de Mon Dec 4 13:07:39 2006 From: bernhard at intevation.de (Bernhard Reiter) Date: Mon, 4 Dec 2006 13:07:39 +0100 Subject: Statist 1.4.1 Released In-Reply-To: <20061202205502.GA2722@localhost.localdomain> References: <20061202205502.GA2722@localhost.localdomain> Message-ID: <200612041307.40565.bernhard@intevation.de> On Saturday 02 December 2006 21:55, Jakson A. Aquino wrote: > Statist 1.4.1 is available! Congratulations! I can see the freshmeat entry was updated as well and people take interest! If you like you could also add a news to wald which then will appear on the front page. Bernhard -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/statist-list/attachments/20061204/7c2b7272/attachment.bin From bernhard at intevation.de Mon Dec 4 13:08:34 2006 From: bernhard at intevation.de (Bernhard Reiter) Date: Mon, 4 Dec 2006 13:08:34 +0100 Subject: Statist 1.4.1 Released In-Reply-To: <200612041307.40565.bernhard@intevation.de> References: <20061202205502.GA2722@localhost.localdomain> <200612041307.40565.bernhard@intevation.de> Message-ID: <200612041308.35256.bernhard@intevation.de> On Monday 04 December 2006 13:07, Bernhard Reiter wrote: > If you like you could also add a news to wald which then will appear > on the front page. Ah, you already did, just approved it. :) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/statist-list/attachments/20061204/33ad8eb7/attachment.bin