From bernhard at intevation.de Thu Jan 3 00:25:27 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Thu, 3 Jan 2008 00:25:27 +0100 Subject: DBFSet_atof_function resolved (again =) In-Reply-To: <47683BFA.2040001@gmail.com> References: <1d5b08270712130115p49293ceq14be71954697fc1d@mail.gmail.com> <200712171509.00016.bernhard@intevation.de> <47683BFA.2040001@gmail.com> Message-ID: <200801030025.32271.bernhard@intevation.de> On Tuesday 18 December 2007 22:30, Bram de Greve wrote: > This issue has been resolved (again) in commit r2798. > This time it has also been fixed upstream. > See libaries/pyshapelib/ChangeLog. Okay, I take it, this should be the version to be merged to trunk for a release? (As you have noticed, I did not manage the christmas release, so we need to make it a welcome2008 release.) ;) -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080103/0e5ec195/attachment.bin From bram.degreve at gmail.com Thu Jan 3 01:59:35 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Thu, 03 Jan 2008 01:59:35 +0100 Subject: DBFSet_atof_function resolved (again =) In-Reply-To: <200801030025.32271.bernhard@intevation.de> References: <1d5b08270712130115p49293ceq14be71954697fc1d@mail.gmail.com> <200712171509.00016.bernhard@intevation.de> <47683BFA.2040001@gmail.com> <200801030025.32271.bernhard@intevation.de> Message-ID: <477C3377.5020503@gmail.com> Bernhard Reiter wrote: > Okay, I take it, this should be the version to be merged to trunk > for a release? > > I don't feel the developments in this branch have been tested well enough for doing a merge yet. Though since WIP-pyhapelib-Unicode branch r2799 it should be feature complete (concerning unicode support for filenames _and_ content). As I mentioned earlier, WIP-pyhapelib-Unicode branch r2795 is the a merge of the trunk (at that time) and WIP-pyshapelib-bramz. As far as I can tell, that merge should be pretty stable. There haven't any changes to WIP-pyshapelib-bramz in an extended period before the merge, and I have been using it myself succesfully during that period (only pyshapelib). ----------------- If you're interested, I've merged the current trunk with the WIP-pyshapelib-bramz branch into something that seems to work (at first glance =). You'll find it as r2795 of the > WIP-pyhapelib-Unicode branch, to make things complicated (*) =) (*) In reality, I will use this as my starting point for further development. Bramz. It should be possible to back port _that_ particular revision to the trunk fairly easily. ----------------- > (As you have noticed, I did not manage the christmas release, so we need to > make it a welcome2008 release.) ;) > > > > ------------------------------------------------------------------------ > > _______________________________________________ > Thuban-devel mailing list > Thuban-devel at intevation.de > https://intevation.de/mailman/listinfo/thuban-devel > From bernhard at intevation.de Thu Jan 3 02:05:21 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Thu, 3 Jan 2008 02:05:21 +0100 Subject: merging back the new pyshapelib Message-ID: <200801030205.25625.bernhard@intevation.de> I've done a (clumsy) attempt to merge back the unicode branch. Here are some results (without much analysis). Bram, if you have comments, let me know. :) I have noticed so far: a) libraries/pyshapelib/setup.py needs an update with new author and version information b) libraries/pyshapelib/README needs an update as we are not using SWIG anymore. Also the NEWS file. c) The test do not seem to be run from the thuban/test/ directory when doing runtests.py d) python libraries/pyshapelib/pytest.py seems to be an unsual name and testing method. Probably we should also use the unittest module. This is what I have done: 1) Fresh checkout of trunk svn checkout svn+ssh://bernhard at scm.wald.intevation.org/thuban/trunk/thuban cd thuban 2) merge attempt taking the last sync point of Bram (r2793) as a reference svn merge svn+ssh://bernhard at scm.wald.intevation.org/thuban/trunk/thuban at 2793 svn+ssh://bernhard at scm.wald.intevation.org/thuban/branches/WIP-pyshapelib-Unicode/thuban . Running the tests I got more trouble then usual: python setup.py build_ext --use-wx-python-swig-hack install_local LANG=de_DE.UTF-8 ====================================================================== ERROR: test_load_1_0.TestNonAsciiColumnName.test ---------------------------------------------------------------------- Traceback (most recent call last): File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/test_load_1_0.py", line 289, in test session = load_session(self.filename()) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/load.py", line 687, in load_session handler.read(filename) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/xmlreader.py", line 75, in read parser.parse(self.__file) File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line 109, in parse xmlreader.IncrementalParser.parse(self, source) File "/usr/lib/python2.4/site-packages/_xmlplus/sax/xmlreader.py", line 123, in parse self.feed(buffer) File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line 216, in feed self._parser.Parse(data, isFinal) File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line 353, in start_element_ns AttributesNSImpl(newattrs, qnames)) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/xmlreader.py", line 114, in startElementNS getattr(self, method_name[0])(name, qname, attrs) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/load.py", line 385, in start_fileshapesource self.idmap[ID] = self.open_shapefile(filename) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/load.py", line 281, in open_shapefile store = self.theSession.OpenShapefile(filename) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/session.py", line 296, in OpenShapefile store = ShapefileStore(self, filename) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/data.py", line 201, in __init__ self.dbftable = table.DBFTable(filename) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/table.py", line 98, in __init__ ftype, name, width, prec = self.dbf.field_info(i) UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 2: ordinal not in range(128) ====================================================================== ERROR: test_load.TestNonAsciiColumnName.test ---------------------------------------------------------------------- Traceback (most recent call last): File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/test_load.py", line 331, in test session = load_session(self.filename()) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/load.py", line 687, in load_session handler.read(filename) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/xmlreader.py", line 75, in read parser.parse(self.__file) File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line 109, in parse xmlreader.IncrementalParser.parse(self, source) File "/usr/lib/python2.4/site-packages/_xmlplus/sax/xmlreader.py", line 123, in parse self.feed(buffer) File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line 216, in feed self._parser.Parse(data, isFinal) File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line 353, in start_element_ns AttributesNSImpl(newattrs, qnames)) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/xmlreader.py", line 114, in startElementNS getattr(self, method_name[0])(name, qname, attrs) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/load.py", line 385, in start_fileshapesource self.idmap[ID] = self.open_shapefile(filename) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/load.py", line 281, in open_shapefile store = self.theSession.OpenShapefile(filename) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/session.py", line 296, in OpenShapefile store = ShapefileStore(self, filename) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/data.py", line 201, in __init__ self.dbftable = table.DBFTable(filename) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/table.py", line 98, in __init__ ftype, name, width, prec = self.dbf.field_info(i) UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 2: ordinal not in range(128) ====================================================================== FAIL: test_transientdb.TestTransientTable.test_auto_transient_table ---------------------------------------------------------------------- Traceback (most recent call last): File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/test_transientdb.py", line 138, in test_auto_transient_table self.run_iceland_political_tests(table) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/test_transientdb.py", line 59, in run_iceland_political_tests self.assertEquals(columns[3].type, FIELDTYPE_INT) AssertionError: 'double' != 'int' ====================================================================== FAIL: test_transientdb.TestTransientTable.test_transient_table ---------------------------------------------------------------------- Traceback (most recent call last): File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/test_transientdb.py", line 111, in test_transient_table self.run_iceland_political_tests(table) File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/test_transientdb.py", line 59, in run_iceland_political_tests self.assertEquals(columns[3].type, FIELDTYPE_INT) AssertionError: 'double' != 'int' ====================================================================== FAIL: Extensions.ogr.test.test_OGRShapestore.TestOGRTable.test_Column ---------------------------------------------------------------------- Traceback (most recent call last): File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Extensions/ogr/test/test_OGRShapestore.py", line 200, in test_Column self.assertEquals(self.table.Column(0).type, FIELDTYPE_INT) AssertionError: 'double' != 'int' ====================================================================== FAIL: Extensions.ogr.test.test_OGRShapestore.TestOGRTable.test_Columns ---------------------------------------------------------------------- Traceback (most recent call last): File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Extensions/ogr/test/test_OGRShapestore.py", line 194, in test_Columns self.assertEquals(self.table.Columns()[0].type, FIELDTYPE_INT) AssertionError: 'double' != 'int' ---------------------------------------------------------------------- Ran 578 tests in 81.882s FAILED (failures=4, errors=2) -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080103/682803d0/attachment.bin From bram.degreve at gmail.com Thu Jan 3 02:07:30 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Thu, 03 Jan 2008 02:07:30 +0100 Subject: testing WIP-pyshapelib-Unicode Message-ID: <477C3552.4010906@gmail.com> Hi all, as far as I can tell, I've implemented all "scheduled" pyshapelib features WIP-pyshapelib-Unicode. As far as pyshapelib is concerned, it now should fully support unicode, both for filenames as textual content. Can you please test this branch? Apart from pyshapelib, it should be identical to the trunk. a few notes: - when creating a new shapefile in Thuban, it now automatically will create an UTF-8 encoded DBF file. You will see this by an accompanying .CPG file. This is necessary, as the UTF-8 encoding cannot be set by the LDID field in the DBF file itself. ArcView should understand this .CPG file though (they're the ones using it ;) - all strings read from shapefiles are now passed to Thuban as unicode strings. But when viewing shapefiles with exotic UTF-8 content (and I mean _exotic_ ;) things bork. Be warned ... Cheers, Bramz From bram.degreve at gmail.com Thu Jan 3 02:10:47 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Thu, 03 Jan 2008 02:10:47 +0100 Subject: merging back the new pyshapelib In-Reply-To: <200801030205.25625.bernhard@intevation.de> References: <200801030205.25625.bernhard@intevation.de> Message-ID: <477C3617.8060004@gmail.com> You're simply too quick. I hardly have sent my requesting for testing, and you already responded =) I'll check this out tomorrow. It's far past bedtime now ;) Bramz Bernhard Reiter wrote: > I've done a (clumsy) attempt to merge back the unicode branch. > Here are some results (without much analysis). > Bram, if you have comments, let me know. :) > > I have noticed so far: > a) libraries/pyshapelib/setup.py needs an update with new author and version > information > b) libraries/pyshapelib/README needs an update as we are not using SWIG > anymore. Also the NEWS file. > c) The test do not seem to be run from the thuban/test/ directory when doing > runtests.py > d) python libraries/pyshapelib/pytest.py seems to be an unsual name and > testing method. Probably we should also use the unittest module. > > This is what I have done: > 1) Fresh checkout of trunk > svn checkout svn+ssh://bernhard at scm.wald.intevation.org/thuban/trunk/thuban > cd thuban > 2) merge attempt taking the last sync point of Bram (r2793) as a reference > svn merge svn+ssh://bernhard at scm.wald.intevation.org/thuban/trunk/thuban at 2793 > svn+ssh://bernhard at scm.wald.intevation.org/thuban/branches/WIP-pyshapelib-Unicode/thuban . > > Running the tests I got more trouble then usual: > python setup.py build_ext --use-wx-python-swig-hack install_local > > LANG=de_DE.UTF-8 > ====================================================================== > ERROR: test_load_1_0.TestNonAsciiColumnName.test > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/test_load_1_0.py", > line 289, in test > session = load_session(self.filename()) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/load.py", > line 687, in load_session > handler.read(filename) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/xmlreader.py", > line 75, in read > parser.parse(self.__file) > File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line > 109, in parse > xmlreader.IncrementalParser.parse(self, source) > File "/usr/lib/python2.4/site-packages/_xmlplus/sax/xmlreader.py", line 123, > in parse > self.feed(buffer) > File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line > 216, in feed > self._parser.Parse(data, isFinal) > File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line > 353, in start_element_ns > AttributesNSImpl(newattrs, qnames)) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/xmlreader.py", > line 114, in startElementNS > getattr(self, method_name[0])(name, qname, attrs) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/load.py", > line 385, in start_fileshapesource > self.idmap[ID] = self.open_shapefile(filename) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/load.py", > line 281, in open_shapefile > store = self.theSession.OpenShapefile(filename) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/session.py", > line 296, in OpenShapefile > store = ShapefileStore(self, filename) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/data.py", > line 201, in __init__ > self.dbftable = table.DBFTable(filename) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/table.py", > line 98, in __init__ > ftype, name, width, prec = self.dbf.field_info(i) > UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 2: > ordinal not in range(128) > > ====================================================================== > ERROR: test_load.TestNonAsciiColumnName.test > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/test_load.py", > line 331, in test > session = load_session(self.filename()) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/load.py", > line 687, in load_session > handler.read(filename) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/xmlreader.py", > line 75, in read > parser.parse(self.__file) > File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line > 109, in parse > xmlreader.IncrementalParser.parse(self, source) > File "/usr/lib/python2.4/site-packages/_xmlplus/sax/xmlreader.py", line 123, > in parse > self.feed(buffer) > File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line > 216, in feed > self._parser.Parse(data, isFinal) > File "/usr/lib/python2.4/site-packages/_xmlplus/sax/expatreader.py", line > 353, in start_element_ns > AttributesNSImpl(newattrs, qnames)) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/xmlreader.py", > line 114, in startElementNS > getattr(self, method_name[0])(name, qname, attrs) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/load.py", > line 385, in start_fileshapesource > self.idmap[ID] = self.open_shapefile(filename) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/load.py", > line 281, in open_shapefile > store = self.theSession.OpenShapefile(filename) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/session.py", > line 296, in OpenShapefile > store = ShapefileStore(self, filename) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/data.py", > line 201, in __init__ > self.dbftable = table.DBFTable(filename) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Thuban/Model/table.py", > line 98, in __init__ > ftype, name, width, prec = self.dbf.field_info(i) > UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 2: > ordinal not in range(128) > > ====================================================================== > FAIL: test_transientdb.TestTransientTable.test_auto_transient_table > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/test_transientdb.py", > line 138, in test_auto_transient_table > self.run_iceland_political_tests(table) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/test_transientdb.py", > line 59, in run_iceland_political_tests > self.assertEquals(columns[3].type, FIELDTYPE_INT) > AssertionError: 'double' != 'int' > > ====================================================================== > FAIL: test_transientdb.TestTransientTable.test_transient_table > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/test_transientdb.py", > line 111, in test_transient_table > self.run_iceland_political_tests(table) > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/test_transientdb.py", > line 59, in run_iceland_political_tests > self.assertEquals(columns[3].type, FIELDTYPE_INT) > AssertionError: 'double' != 'int' > > ====================================================================== > FAIL: Extensions.ogr.test.test_OGRShapestore.TestOGRTable.test_Column > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Extensions/ogr/test/test_OGRShapestore.py", > line 200, in test_Column > self.assertEquals(self.table.Column(0).type, FIELDTYPE_INT) > AssertionError: 'double' != 'int' > > ====================================================================== > FAIL: Extensions.ogr.test.test_OGRShapestore.TestOGRTable.test_Columns > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "/mobilehome/bernhard/hacking/thuban/svn/thuban/test/../Extensions/ogr/test/test_OGRShapestore.py", > line 194, in test_Columns > self.assertEquals(self.table.Columns()[0].type, FIELDTYPE_INT) > AssertionError: 'double' != 'int' > > ---------------------------------------------------------------------- > Ran 578 tests in 81.882s > > FAILED (failures=4, errors=2) > > > > > ------------------------------------------------------------------------ > > _______________________________________________ > Thuban-devel mailing list > Thuban-devel at intevation.de > https://intevation.de/mailman/listinfo/thuban-devel > From bernhard at intevation.de Thu Jan 3 02:12:15 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Thu, 3 Jan 2008 02:12:15 +0100 Subject: DBFSet_atof_function resolved (again =) In-Reply-To: <477C3377.5020503@gmail.com> References: <1d5b08270712130115p49293ceq14be71954697fc1d@mail.gmail.com> <200801030025.32271.bernhard@intevation.de> <477C3377.5020503@gmail.com> Message-ID: <200801030212.16539.bernhard@intevation.de> On Thursday 03 January 2008 01:59, Bram de Greve wrote: > Bernhard Reiter wrote: > > Okay, I take it, this should be the version to be merged to trunk > > for a release? > > > > ? > > I don't feel the developments in this branch have been tested well > enough for doing a merge yet. I was wondering, because after a brief look it seems that the LC_NUMERIC problem is not solved in WIP-pyhapelib-Unicode branch r2795 or was is solved? Best would be to have a version which only change the pyshapelib from swig to a manually coded python api, but keep the functionality as far as possible. > Though since WIP-pyhapelib-Unicode branch > r2799 it should be feature complete (concerning unicode support for > filenames _and_ content). ?As I mentioned earlier, WIP-pyhapelib-Unicode > branch r2795 is the a merge of the trunk (at that time) and > WIP-pyshapelib-bramz. ?As far as I can tell, that merge should be pretty > stable. ?There haven't any changes to WIP-pyshapelib-bramz in an > extended period before the merge, and I have been using it myself > succesfully during that period (only pyshapelib). So you did not do any tests with Thuban? (To find out if your pyshapelib interface really does the same?) -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080103/ad3e2659/attachment.bin From bernhard at intevation.de Thu Jan 3 02:14:11 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Thu, 3 Jan 2008 02:14:11 +0100 Subject: testing WIP-pyshapelib-Unicode In-Reply-To: <477C3552.4010906@gmail.com> References: <477C3552.4010906@gmail.com> Message-ID: <200801030214.12676.bernhard@intevation.de> On Thursday 03 January 2008 02:07, Bram de Greve wrote: > - when creating a new shapefile in Thuban, it now automatically will > create an UTF-8 encoded DBF file. ?You will see this by an accompanying > .CPG file. ?This is necessary, as the UTF-8 encoding cannot be set by > the LDID field in the DBF file itself. ?ArcView should understand this > .CPG file though (they're the ones using it ;) I wonder what should be the default. Is it possible to switch to old behaviour? Are ut-8 dbf files standard with shapefiles now? > - all strings read from shapefiles are now passed to Thuban as unicode > strings. ?But when viewing shapefiles with exotic UTF-8 content (and I > mean _exotic_ ;) things bork. ?Be warned ... What does "bork" mean in this context? We probably should take at least a little effort to not make Thuban break completely when it encounters strange stuff, so where is the problem coming from? Bernhard -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080103/5f13e7ea/attachment.bin From bernhard at intevation.de Thu Jan 3 10:39:58 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Thu, 3 Jan 2008 10:39:58 +0100 Subject: What to release? (wasy: DBFSet_atof_function resolved (again =)) In-Reply-To: <200801030212.16539.bernhard@intevation.de> References: <1d5b08270712130115p49293ceq14be71954697fc1d@mail.gmail.com> <477C3377.5020503@gmail.com> <200801030212.16539.bernhard@intevation.de> Message-ID: <200801031040.02870.bernhard@intevation.de> On Thursday 03 January 2008 02:12, Bernhard Reiter wrote: > On Thursday 03 January 2008 01:59, Bram de Greve wrote: > > Bernhard Reiter wrote: [about /thuban/branches/WIP-pyshapelib-Unicode/thuban at 2798] > > > Okay, I take it, this should be the version to be merged to trunk > > > for a release? ? > > I don't feel the developments in this branch have been tested well > > enough for doing a merge yet. > > I was wondering, because after a brief look it seems that the LC_NUMERIC > problem is not solved in WIP-pyhapelib-Unicode branch r2795 or was is > solved? Best would be to have a version which only change the pyshapelib > from swig to a manually coded python api, but keep the functionality as far > as possible. So I am still wondering which version to release for Thuban 1.2.1 (which I would want to release quite soon). Candidates: a) Not merge the new pyshapelib back to trunk b) use /thuban/branches/WIP-pyshapelib-Unicode/thuban at 2795 Drawback: potentially LC_NUMERIC unstable, which is a regressing less tested then Advantage: More test exposure with the new code. c) use /thuban/branches/WIP-pyshapelib-Unicode/thuban at 2798 2799 or later Drawback: in development, so not tested and ready for a real release. Before christmas b) seemed to be the best guess, I am now wondering if a) is better so we can properly wait until the waves for c) are done. Nothing keeps us from doing another Thuban release once we see fit. What do you think? Bernhard -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080103/b1fc67cb/attachment.bin From dpinte at itae.be Thu Jan 3 11:33:08 2008 From: dpinte at itae.be (Didrik Pinte) Date: Thu, 03 Jan 2008 11:33:08 +0100 Subject: What to release? (wasy: DBFSet_atof_function resolved (again =)) In-Reply-To: <200801031040.02870.bernhard@intevation.de> References: <1d5b08270712130115p49293ceq14be71954697fc1d@mail.gmail.com> <477C3377.5020503@gmail.com> <200801030212.16539.bernhard@intevation.de> <200801031040.02870.bernhard@intevation.de> Message-ID: <1199356388.5440.71.camel@ddp.simpson> On Thu, 2008-01-03 at 10:39 +0100, Bernhard Reiter wrote: > So I am still wondering which version to release for Thuban 1.2.1 > (which I would want to release quite soon). > Candidates: > a) Not merge the new pyshapelib back to trunk > b) use /thuban/branches/WIP-pyshapelib-Unicode/thuban at 2795 > Drawback: potentially LC_NUMERIC unstable, which is a regressing > less tested then > Advantage: More test exposure with the new code. > c) use /thuban/branches/WIP-pyshapelib-Unicode/thuban at 2798 2799 or later > Drawback: in development, so not tested and ready for a real release. > > Before christmas b) seemed to be the best guess, I am now wondering if a) > is better so we can properly wait until the waves for c) are done. > Nothing keeps us from doing another Thuban release once we see fit. > > What do you think? I think a) is the best. The LC_NUMERIC problem is really annoying and cause strange things for the end user. Thus, I go for a) and c) as soon as it is stable enough (that can be very soon ;-)). Didrik -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080103/253782e4/attachment.bin From bram.degreve at gmail.com Thu Jan 3 12:29:36 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Thu, 03 Jan 2008 12:29:36 +0100 Subject: testing WIP-pyshapelib-Unicode In-Reply-To: <200801030214.12676.bernhard@intevation.de> References: <477C3552.4010906@gmail.com> <200801030214.12676.bernhard@intevation.de> Message-ID: <477CC720.3090600@gmail.com> Bernhard Reiter wrote: > On Thursday 03 January 2008 02:07, Bram de Greve wrote: > >> - when creating a new shapefile in Thuban, it now automatically will >> create an UTF-8 encoded DBF file. You will see this by an accompanying >> .CPG file. This is necessary, as the UTF-8 encoding cannot be set by >> the LDID field in the DBF file itself. ArcView should understand this >> .CPG file though (they're the ones using it ;) >> > > I wonder what should be the default. > The user should be able to configure this in Thuban. The default probably should be LDID/87 (= 0x57). That's an ANSI code page. > Is it possible to switch to old behaviour? > Sure. I merely choose UTF-8 in the Unicode branch because I'm testing UTF-8 at the moment. We can switch back to any other behaviour in Thuban/Model/table.py > Are ut-8 dbf files standard with shapefiles now? > No, a default installation will use Windows ANSI. There's a catch though. Windows ANSI is not uniquely defined. It could be any of the CP125x pages, whatever is set by the box. At least that's what I think. I'm not sure about it, an expert would need to confirm this. At any rate, wherever Windows ANSI or just ANSI is mentioned, I assume CP1252, as that's the most common mentioned ANSI page. (0x3 and 0x57 are both set to CP1252 in pyshapelib). > >> - all strings read from shapefiles are now passed to Thuban as unicode >> strings. But when viewing shapefiles with exotic UTF-8 content (and I >> mean _exotic_ ;) things bork. Be warned ... >> > > What does "bork" mean in this context? > We probably should take at least a little effort to not make Thuban break > completely when it encounters strange stuff, so where is the problem coming > from? > "to bork" means that Thuban is not completely prepared to handle exotic unicode characters. Thuban/Model/table.py now returns unicode strings exclusively, and I don't think that's where the problem is caused. When I try to display the table, things go really wrong. I must prepare a new nice test shapefile so you can see yourself. I tried this a month ago, but I've lost the testfile =) I'll do this later today. Bramz > Bernhard > > > ------------------------------------------------------------------------ > > _______________________________________________ > Thuban-devel mailing list > Thuban-devel at intevation.de > https://intevation.de/mailman/listinfo/thuban-devel > From bram.degreve at gmail.com Thu Jan 3 13:34:28 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Thu, 03 Jan 2008 13:34:28 +0100 Subject: merging back the new pyshapelib In-Reply-To: <200801030205.25625.bernhard@intevation.de> References: <200801030205.25625.bernhard@intevation.de> Message-ID: <477CD654.8060603@gmail.com> Bernhard Reiter wrote: > Running the tests I got more trouble then usual: > python setup.py build_ext --use-wx-python-swig-hack install_local > > OK, the troubles must be caused by my code, as testing the trunk gives much saner results =) I still have those permission denieds though, but apparently that's just part of the test as it seems to have succeeded. .............SSS................................................................ ...................................................SS........Exception exception s.OSError: (13, 'Permission denied', 'c:\\docume~1\\bramz\\locals~1\\temp\\tmpv9 -fof\\transientdb') in > ignored Exception exceptions.OSError: (41, 'Directory not empty', 'c:\\docume~1\\bramz\\ locals~1\\temp\\tmpv9-fof') in > ignored .Exception exceptions.OSError: (13, 'Permission denied', 'c:\\docume~1\\bramz\\l ocals~1\\temp\\tmpcy8um0\\transientdb') in > ignored Exception exceptions.OSError: (41, 'Directory not empty', 'c:\\docume~1\\bramz\\ locals~1\\temp\\tmpcy8um0') in > ignored ...........SSSS.....................SSSS..........SSSS.......................... ..............SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS..SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS SSSSSSS...S..................................................................... .........................................SSS.................................... ..............SSSSSSSSSSSSSSSSSSSSSS From bernhard at intevation.de Thu Jan 3 14:32:54 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Thu, 3 Jan 2008 14:32:54 +0100 Subject: merging back the new pyshapelib In-Reply-To: <477CD654.8060603@gmail.com> References: <200801030205.25625.bernhard@intevation.de> <477CD654.8060603@gmail.com> Message-ID: <200801031432.55183.bernhard@intevation.de> On Thursday 03 January 2008 13:34, Bram de Greve wrote: > Bernhard Reiter wrote: > > Running the tests I got more trouble then usual: > > ?python setup.py ?build_ext --use-wx-python-swig-hack ?install_local > > > > ? > > OK, the troubles must be caused by my code, as testing the trunk gives > much saner results =) Some of the failure are very likely to be caused by your changes. I suggest that you at least make sure you can run the Thuban tests. Only the two OGR failures are to be expected currently (see the release notes). > I still have those permission denieds though, but apparently that's just > part of the test as it seems to have succeeded. > > .............SSS........................................................... >..... ...................................................SS........Exception > exception > s.OSError: (13, 'Permission denied', > 'c:\\docume~1\\bramz\\locals~1\\temp\\tmpv9 What are the skipped tests? Postgis? There are some permission denied tests, but not that many and usually they do not print something on stdout. So which test is it? (See the Readme in the test/ how to run single tests.) Bernhard -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080103/7ce36218/attachment.bin From bram.degreve at gmail.com Thu Jan 3 14:40:16 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Thu, 03 Jan 2008 14:40:16 +0100 Subject: What to release? (wasy: DBFSet_atof_function resolved (again =)) In-Reply-To: <200801031040.02870.bernhard@intevation.de> References: <1d5b08270712130115p49293ceq14be71954697fc1d@mail.gmail.com> <477C3377.5020503@gmail.com> <200801030212.16539.bernhard@intevation.de> <200801031040.02870.bernhard@intevation.de> Message-ID: <477CE5C0.9070207@gmail.com> Bernhard Reiter wrote: > On Thursday 03 January 2008 02:12, Bernhard Reiter wrote: > >> >> I was wondering, because after a brief look it seems that the LC_NUMERIC >> problem is not solved in WIP-pyhapelib-Unicode branch r2795 or was is >> solved? Best would be to have a version which only change the pyshapelib >> from swig to a manually coded python api, but keep the functionality as far >> as possible. >> > > Indeed, the LC_NUMERIC fix disappeared when I rewrote dbflibmodule.c, about a year ago in WIP-pyshapelib-Bramz. However, it should only be a matter of calling DBFSet_atof_function again in initdbflib. Bernhard: "So you did not do any tests with Thuban? (To find out if your pyshapelib interface really does the same?)" No I didn't. All I can tell is that I've been using the pyshapelib library of WIP-pyshapelib-Bramz at 2755 in our own lab and software without troubles. As far as I can tell from my own use, it has the same interface as before and it never crashed. > So I am still wondering which version to release for Thuban 1.2.1 > (which I would want to release quite soon). > Candidates: > a) Not merge the new pyshapelib back to trunk > b) use /thuban/branches/WIP-pyshapelib-Unicode/thuban at 2795 > Drawback: potentially LC_NUMERIC unstable, which is a regressing > less tested then > Advantage: More test exposure with the new code. > c) use /thuban/branches/WIP-pyshapelib-Unicode/thuban at 2798 2799 or later > Drawback: in development, so not tested and ready for a real release. > > Before christmas b) seemed to be the best guess, I am now wondering if a) > is better so we can properly wait until the waves for c) are done. > Nothing keeps us from doing another Thuban release once we see fit. > > What do you think? > > Bernhard > > > Here's a suggestion: * What do we know? - WIP-pyshapelib-Unicode at 2795 is a breaindead merge between of trunk @ 2793 (as it still stands today) and WIP-pyshapelib-Bramz @ (2734:2755]. The latter has a stable pyshapelib (tested myself as I've been using it over the past year without problems). This however, misses the DBFSet_atof_function in initdbflib. - There's no need to add DBFSet_atof_function to the WIP-pyshapelib-Unicode branch, as that branch is using another fix already @ 2798. * What can we do? - make a final addition to the WIP-pyshapelib-Bramz with only the DBFSet_atof_function fix in initdbflib.c. We would arrive at rev 2800. - merge WIP-pyshapelib-Bramz rev (2734:2800] with trunk. This should be straight forward. It was when I did before. - Virtually we are at the same point as WIP-pyshapelib-Unicode at 2795 with the additional DBFSet_atof_function fix. - if it passes the test, release. Otherwise, back off and go back to plan A? ON SECOND THOUGHT: Forget about my suggestion, because I've just somewhat tried it myself, and it borks =) Go for plan A ;) At any rate, the head of WIP-pyshapelib-Unicode is not ready for release. We're not far anymore, but there's still some work to do. Bramz From bram.degreve at gmail.com Thu Jan 3 14:50:57 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Thu, 03 Jan 2008 14:50:57 +0100 Subject: merging back the new pyshapelib In-Reply-To: <200801031432.55183.bernhard@intevation.de> References: <200801030205.25625.bernhard@intevation.de> <477CD654.8060603@gmail.com> <200801031432.55183.bernhard@intevation.de> Message-ID: <477CE841.4080404@gmail.com> Bernhard Reiter wrote: > On Thursday 03 January 2008 13:34, Bram de Greve wrote: > >> Bernhard Reiter wrote: >> >>> Running the tests I got more trouble then usual: >>> python setup.py build_ext --use-wx-python-swig-hack install_local >>> >>> >>> >> OK, the troubles must be caused by my code, as testing the trunk gives >> much saner results =) >> > > Some of the failure are very likely to be caused by your changes. > I suggest that you at least make sure you can run the Thuban tests. > Only the two OGR failures are to be expected currently (see the release > notes). > > >> I still have those permission denieds though, but apparently that's just >> part of the test as it seems to have succeeded. >> >> .............SSS........................................................... >> ..... ...................................................SS........Exception >> exception >> s.OSError: (13, 'Permission denied', >> 'c:\\docume~1\\bramz\\locals~1\\temp\\tmpv9 >> > > What are the skipped tests? Postgis? > There are some permission denied tests, but not that many and usually they do > not print something on stdout. So which test is it? > (See the Readme in the test/ how to run single tests.) > > The permission denieds seem to be from test_load.py. The skipped tests are gdal tests (no gdal support apparently) and postgis. Also, I've just tested the head of WIP-pyshapelib-branch again, and it - apart from the permission denieds and skipped tests - the only difference with the trunk seems to be two errors on TestNonAsciiColumnNames in test_load.py and test_load_1_0.py ====================================================================== ERROR: test_load_1_0.TestNonAsciiColumnName.test ---------------------------------------------------------------------- Traceback (most recent call last): File "D:\bram\thuban_work\WIP-pyshapelib-bramz\test\test_load_1_0.py", line 27 9, in test dbf.add_field('Fl\xe4che', dbflib.FTDouble, 10, 5) File "C:\Python24\lib\encodings\cp1252.py", line 18, in encode return codecs.charmap_encode(input,errors,encoding_map) UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 2: ordinal not in range(128) > Bernhard > > > ------------------------------------------------------------------------ > > _______________________________________________ > Thuban-devel mailing list > Thuban-devel at intevation.de > https://intevation.de/mailman/listinfo/thuban-devel > From bram.degreve at gmail.com Thu Jan 3 20:12:40 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Thu, 03 Jan 2008 20:12:40 +0100 Subject: pyshapelib homepage Message-ID: <477D33A8.9040009@gmail.com> I've found this in the libraries/pyshapelib/README: The bindings themselves don't have a homepage at the moment, but the source tarballs/zip files can be downloaded from http://ftp.intevation.de/users/bh/pyshapelib/ With a new release of pyshapelib finally taking shape, is it much work to set up a homepage for pyshapelib? It doesn't have to be much, but it would be much nicer as reference. Cheers, Bramz From bernhard at intevation.de Fri Jan 4 11:08:07 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Fri, 4 Jan 2008 11:08:07 +0100 Subject: pyshapelib homepage In-Reply-To: <477D33A8.9040009@gmail.com> References: <477D33A8.9040009@gmail.com> Message-ID: <200801041108.13414.bernhard@intevation.de> On Thursday 03 January 2008 20:12, Bram de Greve wrote: > With a new release of pyshapelib finally taking shape, is it much work > to set up a homepage for pyshapelib? ?It doesn't have to be much, but it > would be much nicer as reference. We can do it, of course. Please apply for a project on wald.intevation.org for pyshapelib. (I need to authorize it later.) There is a documentation for uploading pages for a homepage then. So someone would need to make them. :) Best, Bernhard -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080104/9775b0d7/attachment.bin From bernhard at intevation.de Fri Jan 4 11:12:50 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Fri, 4 Jan 2008 11:12:50 +0100 Subject: What to release? (wasy: DBFSet_atof_function resolved (again =)) In-Reply-To: <477CE5C0.9070207@gmail.com> References: <1d5b08270712130115p49293ceq14be71954697fc1d@mail.gmail.com> <200801031040.02870.bernhard@intevation.de> <477CE5C0.9070207@gmail.com> Message-ID: <200801041112.51924.bernhard@intevation.de> On Thursday 03 January 2008 14:40, Bram de Greve wrote: > Bernhard Reiter wrote: > > On Thursday 03 January 2008 02:12, Bernhard Reiter wrote: > >> I was wondering, because after a brief look it seems that the LC_NUMERIC > >> problem is not solved in WIP-pyhapelib-Unicode branch r2795 or was is > >> solved? Best would be to have a version which only change the pyshapelib > >> from swig to a manually coded python api, but keep the functionality as > >> far as possible. > > Indeed, the LC_NUMERIC fix disappeared when I rewrote dbflibmodule.c, > about a year ago in WIP-pyshapelib-Bramz. However, it should only be a > matter of calling DBFSet_atof_function again in initdbflib. Okay, this sounds easy, I might look into trying this next time if I had not read below that you had tried and it did not work. > Bernhard: "So you did not do any tests with Thuban? > (To find out if your pyshapelib interface really does the same?)" > > > No I didn't. All I can tell is that I've been using the pyshapelib > library of WIP-pyshapelib-Bramz at 2755 in our own lab and software > without troubles. As far as I can tell from my own use, it has the same > interface as before and it never crashed. Okay, some tests with Thuban would be been cool. At least running the test suite, as we do now. > > So I am still wondering which version to release for Thuban 1.2.1 > > (which I would want to release quite soon). > > Candidates: > > a) Not merge the new pyshapelib back to trunk > > b) use /thuban/branches/WIP-pyshapelib-Unicode/thuban at 2795 > > Drawback: potentially LC_NUMERIC unstable, which is a regressing > > less tested then > > Advantage: More test exposure with the new code. > > c) use /thuban/branches/WIP-pyshapelib-Unicode/thuban at 2798 2799 or later > > Drawback: in development, so not tested and ready for a real release. > > > > Before christmas b) seemed to be the best guess, I am now wondering if a) > > is better so we can properly wait until the waves for c) are done. > > Nothing keeps us from doing another Thuban release once we see fit. > Here's a suggestion: > * What do we know? > - WIP-pyshapelib-Unicode at 2795 is a breaindead merge between of trunk @ > 2793 (as it still stands today) and WIP-pyshapelib-Bramz @ (2734:2755]. > The latter has a stable pyshapelib (tested myself as I've been using it > over the past year without problems). This however, misses the > DBFSet_atof_function in initdbflib. > - There's no need to add DBFSet_atof_function to the > WIP-pyshapelib-Unicode branch, as that branch is using another fix > already @ 2798. > * What can we do? > - make a final addition to the WIP-pyshapelib-Bramz with only the > DBFSet_atof_function fix in initdbflib.c. We would arrive at rev 2800. > - merge WIP-pyshapelib-Bramz rev (2734:2800] with trunk. This should be > straight forward. It was when I did before. > - Virtually we are at the same point as WIP-pyshapelib-Unicode at 2795 with > the additional DBFSet_atof_function fix. > - if it passes the test, release. Otherwise, back off and go back to > plan A? > > > ON SECOND THOUGHT: Forget about my suggestion, because I've just somewhat > tried it myself, and it borks =) Go for plan A ;) Okay. > At any rate, the head of WIP-pyshapelib-Unicode is not ready for release. > We're not far anymore, but there's still some work to do. Wonderful, it is better to not obstruct this fine work with the pressure of a release then. Bernhard -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080104/51a980de/attachment.bin From bram.degreve at gmail.com Fri Jan 4 11:45:04 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Fri, 04 Jan 2008 11:45:04 +0100 Subject: pyshapelib homepage In-Reply-To: <200801041108.13414.bernhard@intevation.de> References: <477D33A8.9040009@gmail.com> <200801041108.13414.bernhard@intevation.de> Message-ID: <477E0E30.4000400@gmail.com> Bernhard Reiter wrote: > On Thursday 03 January 2008 20:12, Bram de Greve wrote: > >> With a new release of pyshapelib finally taking shape, is it much work >> to set up a homepage for pyshapelib? It doesn't have to be much, but it >> would be much nicer as reference. >> > > We can do it, of course. > Please apply for a project on wald.intevation.org for pyshapelib. > (I need to authorize it later.) > Seems I'm not allowed to register any projects. Setting up an separate project for pyshapelib does not interfere with its source being in the Thuban tree, does it? > There is a documentation for uploading pages for a homepage then. > So someone would need to make them. :) > > Best, > Bernhard > > > ------------------------------------------------------------------------ > > _______________________________________________ > Thuban-devel mailing list > Thuban-devel at intevation.de > https://intevation.de/mailman/listinfo/thuban-devel > From bram.degreve at gmail.com Fri Jan 4 12:07:04 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Fri, 04 Jan 2008 12:07:04 +0100 Subject: merging back the new pyshapelib In-Reply-To: <200801041115.22354.bernhard@intevation.de> References: <200801030205.25625.bernhard@intevation.de> <477CD009.50208@gmail.com> <200801041115.22354.bernhard@intevation.de> Message-ID: <477E1358.4050707@gmail.com> Bernhard Reiter wrote: >>> Running the tests I got more trouble then usual: >>> python setup.py build_ext --use-wx-python-swig-hack install_local >>> >>> >> Well, I tried this as well on my windoze. I'm supposed to run >> test/runtests.py ? I've got like a zillion of permission denieds though :( >> Scary. I can't believe this is all due to pyshapelib though. >> > > We got this followed up already, so of it was caused by the new pyshapelib. > Also when compiling on Debian sid I got quite a few compiler warning, > did you check them as well? > > I've looked a bit further into it, and it boils down to two distinct errors (apart from the permission denieds): !!! This is about the WIP-pyshapelib-Unicode branch! ====================================================================== FAIL: test_load.TestNonAsciiColumnName.test ---------------------------------------------------------------------- Traceback (most recent call last): File "D:\bram\thuban_work\WIP-pyshapelib-Unicode\thuban\test\test_load.py", li ne 339, in test self.fail("Cannot load file with non-ascii column names") AssertionError: Cannot load file with non-ascii column names ====================================================================== FAIL: test_load_1_0.TestNonAsciiColumnName.test ---------------------------------------------------------------------- Traceback (most recent call last): File "D:\bram\thuban_work\WIP-pyshapelib-Unicode\thuban\test\test_load_1_0.py" , line 297, in test self.fail("Cannot load file with non-ascii column names") AssertionError: Cannot load file with non-ascii column names This is about the column name Fl?che. It gets into dbflib correctly, dbflib returns it correctly, but then when it is written to the XML for testing, it is not UTF-8 encoded to 'Fl\xc3\xa4che'. Instead it is written as Fl?che, failing the test. Might it be caused by the fact that dbflib is now set to return unicode strings? ====================================================================== FAIL: test_transientdb.TestTransientTable.test_auto_transient_table ---------------------------------------------------------------------- Traceback (most recent call last): File "D:\bram\thuban_work\WIP-pyshapelib-Unicode\thuban\test\test_transientdb. py", line 138, in test_auto_transient_table self.run_iceland_political_tests(table) File "D:\bram\thuban_work\WIP-pyshapelib-Unicode\thuban\test\test_transientdb. py", line 59, in run_iceland_political_tests self.assertEquals(columns[3].type, FIELDTYPE_INT) AssertionError: 'double' != 'int' ====================================================================== FAIL: test_transientdb.TestTransientTable.test_transient_table ---------------------------------------------------------------------- Traceback (most recent call last): File "D:\bram\thuban_work\WIP-pyshapelib-Unicode\thuban\test\test_transientdb. py", line 111, in test_transient_table self.run_iceland_political_tests(table) File "D:\bram\thuban_work\WIP-pyshapelib-Unicode\thuban\test\test_transientdb. py", line 59, in run_iceland_political_tests self.assertEquals(columns[3].type, FIELDTYPE_INT) AssertionError: 'double' != 'int' This is caused by a change in the handling of numerical fields in the shapelib C library. Integer fields of more than 10 characters (as the third column in that file is), are returned as doubles to avoid an overflow. So, we shall either have to change the test, or change policital.dbf. I don't have access to a linux box ATM, so I can't look into these warnings. But there are a few on windows as well. I'll see to fix them. Bramz From bh at intevation.de Fri Jan 4 12:40:55 2008 From: bh at intevation.de (Bernhard Herzog) Date: Fri, 4 Jan 2008 12:40:55 +0100 Subject: merging back the new pyshapelib In-Reply-To: <477E1358.4050707@gmail.com> References: <200801030205.25625.bernhard@intevation.de> <200801041115.22354.bernhard@intevation.de> <477E1358.4050707@gmail.com> Message-ID: <200801041240.58679.bh@intevation.de> On Friday 04 January 2008 12:07, Bram de Greve wrote: > This is caused by a change in the handling of numerical fields in the > shapelib C library. Integer fields of more than 10 characters (as the > third column in that file is), are returned as doubles to avoid an > overflow. Wouldn't it make more sense to return them as long int instead of double at the python level? Bernhard -- Bernhard Herzog Intevation GmbH, Osnabr?ck Amtsgericht Osnabr?ck, HR B 18998 http://www.intevation.de/ Gesch?ftsf?hrer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080104/9362498c/attachment.bin From bernhard at intevation.de Fri Jan 4 14:51:39 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Fri, 4 Jan 2008 14:51:39 +0100 Subject: pyshapelib homepage In-Reply-To: <477E0E30.4000400@gmail.com> References: <477D33A8.9040009@gmail.com> <200801041108.13414.bernhard@intevation.de> <477E0E30.4000400@gmail.com> Message-ID: <200801041451.44502.bernhard@intevation.de> On Friday 04 January 2008 11:45, Bram de Greve wrote: > Seems I'm not allowed to register any projects. Okay, I though you could apply for them which would need authorisation, but it might be that we switched this off. I could register it. > Setting up an separate project for pyshapelib does not interfere with > its source being in the Thuban tree, does it? No, though for the long run we could think about moving it into its own repostory then. On the other hand, we would produce a homepage for pyshapelib on thuban.wald.intevation.org/pyshapelib right now within the thuban project on wald, if you prefer this. Would be less overhead. > > There is a documentation for uploading pages for a homepage then. > > So someone would need to make them. :) -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080104/8c540d67/attachment.bin From bernhard at intevation.de Fri Jan 4 14:54:56 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Fri, 4 Jan 2008 14:54:56 +0100 Subject: testfailures on WIP-pyshapelib-Unicode branch (was: merging back the new pyshapelib) In-Reply-To: <477E1358.4050707@gmail.com> References: <200801030205.25625.bernhard@intevation.de> <200801041115.22354.bernhard@intevation.de> <477E1358.4050707@gmail.com> Message-ID: <200801041454.57001.bernhard@intevation.de> On Friday 04 January 2008 12:07, Bram de Greve wrote: > ====================================================================== > FAIL: test_load_1_0.TestNonAsciiColumnName.test > ---------------------------------------------------------------------- > Traceback (most recent call last): > ? File > "D:\bram\thuban_work\WIP-pyshapelib-Unicode\thuban\test\test_load_1_0.py" , > line 297, in test > ? ? self.fail("Cannot load file with non-ascii column names") > AssertionError: Cannot load file with non-ascii column names > > > > This is about the column name Fl?che. It gets into dbflib correctly, > dbflib returns it correctly, but then when it is written to the XML for > testing, it is not UTF-8 encoded to > 'Fl\xc3\xa4che'. Instead it is written as Fl?che, failing the test. > Might it be caused by the fact that dbflib is now set to return unicode > strings? Maybe something else needs a change, it possibly is related to this area. -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080104/d8f13869/attachment.bin From dpinte at itae.be Sun Jan 6 09:51:26 2008 From: dpinte at itae.be (Didrik Pinte) Date: Sun, 06 Jan 2008 09:51:26 +0100 Subject: merging back the new pyshapelib In-Reply-To: <477E1358.4050707@gmail.com> References: <200801030205.25625.bernhard@intevation.de> <477CD009.50208@gmail.com> <200801041115.22354.bernhard@intevation.de> <477E1358.4050707@gmail.com> Message-ID: <1199609486.22316.7.camel@ddp.simpson> On Fri, 2008-01-04 at 12:07 +0100, Bram de Greve wrote: > > I don't have access to a linux box ATM, so I can't look into these > warnings. But there are a few on windows as well. I'll see to fix > them. > > Bramz Hi, It seems to be exactly the same on a linux box. See the attached log. Didrik -------------- next part -------------- A non-text attachment was scrubbed... Name: debian_sid_test.log Type: text/x-log Size: 19212 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080106/0215de5d/debian_sid_test.log -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080106/0215de5d/attachment.bin From dpinte at itae.be Sun Jan 6 10:09:04 2008 From: dpinte at itae.be (Didrik Pinte) Date: Sun, 06 Jan 2008 10:09:04 +0100 Subject: Extension wms.py should move to owslib In-Reply-To: <200712111008.52935.bernhard@intevation.de> References: <200712111008.52935.bernhard@intevation.de> Message-ID: <1199610544.22316.12.camel@ddp.simpson> On Tue, 2007-12-11 at 10:08 +0100, Bernhard Reiter wrote: > Deal Thuban-Developers, > > after poking around in wms.py (because it is somewhat broken), > Didrik and I sketched the plan to move it to owslib. > http://trac.gispython.org/projects/PCL/wiki/OwsLib > > The reason is that pyogclib is abandoned (and broken) > and owslib is active. > Whoever gets first to it, can do the move, but we need to pay attention > that owslib currently is changing its interface - so we might need > to wait for rel-0.3.0. I've started some work on that (debian packaging of owslib and updating the old wms to the latest svn code of owslib). Didrik -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080106/7985e62c/attachment.bin From bernhard at intevation.de Mon Jan 7 12:45:56 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Mon, 7 Jan 2008 12:45:56 +0100 Subject: merging back the new pyshapelib In-Reply-To: <1199609486.22316.7.camel@ddp.simpson> References: <200801030205.25625.bernhard@intevation.de> <477E1358.4050707@gmail.com> <1199609486.22316.7.camel@ddp.simpson> Message-ID: <200801071246.01184.bernhard@intevation.de> On Sunday 06 January 2008 09:51, Didrik Pinte wrote: > On Fri, 2008-01-04 at 12:07 +0100, Bram de Greve wrote: > > I don't have access to a linux box ATM, so I can't look into these > > warnings. ?But there are a few on windows as well. ?I'll see to fix > > them. > It seems to be exactly the same on a linux box. See the attached log. I think this refered to the compliation "warning"s, not the test failures. :) -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080107/0c3415c4/attachment.bin From bernhard at intevation.de Mon Jan 7 12:46:19 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Mon, 7 Jan 2008 12:46:19 +0100 Subject: Extension wms.py should move to owslib In-Reply-To: <1199610544.22316.12.camel@ddp.simpson> References: <200712111008.52935.bernhard@intevation.de> <1199610544.22316.12.camel@ddp.simpson> Message-ID: <200801071246.20750.bernhard@intevation.de> On Sunday 06 January 2008 10:09, Didrik Pinte wrote: > I've started some work on that (debian packaging of owslib and updating > the old wms to the latest svn code of owslib). Cool! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080107/2fbeb0b4/attachment.bin From bram.degreve at gmail.com Mon Jan 7 15:03:13 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Mon, 07 Jan 2008 15:03:13 +0100 Subject: pyshapelib homepage In-Reply-To: <200801041451.44502.bernhard@intevation.de> References: <477D33A8.9040009@gmail.com> <200801041108.13414.bernhard@intevation.de> <477E0E30.4000400@gmail.com> <200801041451.44502.bernhard@intevation.de> Message-ID: <47823121.2000400@gmail.com> Bernhard Reiter wrote: > On Friday 04 January 2008 11:45, Bram de Greve wrote: > >> Seems I'm not allowed to register any projects. >> > > Okay, I though you could apply for them which would need authorisation, > but it might be that we switched this off. > I could register it. > > >> Setting up an separate project for pyshapelib does not interfere with >> its source being in the Thuban tree, does it? >> > > No, though for the long run we could think about moving it into its own > repostory then. On the other hand, we would produce a homepage > for pyshapelib on thuban.wald.intevation.org/pyshapelib right now > within the thuban project on wald, if you prefer this. > Would be less overhead. > For now, I think a page within the thuban project is sufficient. I don't think there's much more than one page to fill anyway ... Bram > > >>> There is a documentation for uploading pages for a homepage then. >>> So someone would need to make them. :) >>> > > > ------------------------------------------------------------------------ > > _______________________________________________ > Thuban-devel mailing list > Thuban-devel at intevation.de > https://intevation.de/mailman/listinfo/thuban-devel > From bram.degreve at gmail.com Mon Jan 7 15:07:23 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Mon, 07 Jan 2008 15:07:23 +0100 Subject: merging back the new pyshapelib In-Reply-To: <200801041240.58679.bh@intevation.de> References: <200801030205.25625.bernhard@intevation.de> <200801041115.22354.bernhard@intevation.de> <477E1358.4050707@gmail.com> <200801041240.58679.bh@intevation.de> Message-ID: <4782321B.5050106@gmail.com> Bernhard Herzog wrote: > On Friday 04 January 2008 12:07, Bram de Greve wrote: > >> This is caused by a change in the handling of numerical fields in the >> shapelib C library. Integer fields of more than 10 characters (as the >> third column in that file is), are returned as doubles to avoid an >> overflow. >> > > Wouldn't it make more sense to return them as long int instead of double at > the python level? > > So you mean from C double back to python long int? Perhaps. There's an API function that does exactly that: PyLong_FromDouble. Would it be better to return all integers as long ints in that case? Bram > ------------------------------------------------------------------------ > > _______________________________________________ > Thuban-devel mailing list > Thuban-devel at intevation.de > https://intevation.de/mailman/listinfo/thuban-devel > From bram.degreve at gmail.com Mon Jan 7 15:08:39 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Mon, 07 Jan 2008 15:08:39 +0100 Subject: merging back the new pyshapelib In-Reply-To: <200801071246.01184.bernhard@intevation.de> References: <200801030205.25625.bernhard@intevation.de> <477E1358.4050707@gmail.com> <1199609486.22316.7.camel@ddp.simpson> <200801071246.01184.bernhard@intevation.de> Message-ID: <47823267.2080306@gmail.com> Bernhard Reiter wrote: > On Sunday 06 January 2008 09:51, Didrik Pinte wrote: > >> On Fri, 2008-01-04 at 12:07 +0100, Bram de Greve wrote: >> >>> I don't have access to a linux box ATM, so I can't look into these >>> warnings. But there are a few on windows as well. I'll see to fix >>> them. >>> > > >> It seems to be exactly the same on a linux box. See the attached log. >> > > I think this refered to the compliation "warning"s, > not the test failures. :) > > Yeah, it's about compiler warnings. But thanks this info anyway! It's most useful. Bram > > > ------------------------------------------------------------------------ > > _______________________________________________ > Thuban-devel mailing list > Thuban-devel at intevation.de > https://intevation.de/mailman/listinfo/thuban-devel > From bernhard at intevation.de Mon Jan 7 17:42:51 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Mon, 7 Jan 2008 17:42:51 +0100 Subject: pyshapelib homepage In-Reply-To: <47823121.2000400@gmail.com> References: <477D33A8.9040009@gmail.com> <200801041451.44502.bernhard@intevation.de> <47823121.2000400@gmail.com> Message-ID: <200801071742.52647.bernhard@intevation.de> On Monday 07 January 2008 15:03, Bram de Greve wrote: > For now, I think a page within the thuban project is sufficient. ?I > don't think there's much more than one page to fill anyway ... Okay, feel free to upload it, using the instructions on wald. :)= -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080107/fa13a618/attachment.bin From bernhard at intevation.de Tue Jan 8 14:19:53 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Tue, 8 Jan 2008 14:19:53 +0100 Subject: [thuban-Bugs][571] Error when adding a layer In-Reply-To: <20071230183205.A6AF640507@pyrosoma.intevation.org> References: <20071230183205.A6AF640507@pyrosoma.intevation.org> Message-ID: <200801081419.57058.bernhard@intevation.de> Didrik, this crash is strange, if you happen to have a Thuban on Windows, could you try just to open the hs1-1500.shp file with it? (Please write feedback into the tracker on wald, thanks!) Bernhard On Sunday 30 December 2007 19:32, thuban-bugs at wald.intevation.org wrote: > Traceback (most recent call last): > ? File "/usr/lib/thuban/Thuban/UI/view.py", line 190, in OnIdle > ? ? event.RequestMore(self._do_redraw()) > ? File "/usr/lib/python2.4/site-packages/wx-2.6-gtk2-unicode/wx/_core.py", > line 6749, in RequestMore return _core_.IdleEvent_RequestMore(*args, > **kwargs) > MemoryError: argument number 1: > > I am using Thuban version: Thuban Release Version 1.2.0 > ?on my Debian Sid. The layer that nade it crash was file hs1-1500.shp from > http://www.juntadeandalucia.es/obraspublicasytransportes/www/estaticas/cart >ografia/descargas/archivos/hidrografialineal.zip -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080108/6b9255d7/attachment.bin From dpinte at itae.be Tue Jan 8 14:41:33 2008 From: dpinte at itae.be (Didrik Pinte) Date: Tue, 08 Jan 2008 14:41:33 +0100 Subject: [thuban-Bugs][571] Error when adding a layer In-Reply-To: <200801081419.57058.bernhard@intevation.de> References: <20071230183205.A6AF640507@pyrosoma.intevation.org> <200801081419.57058.bernhard@intevation.de> Message-ID: <1199799693.5141.12.camel@ddp.simpson> I plan this before the end of the week. Very strange bug. Didrik On Tue, 2008-01-08 at 14:19 +0100, Bernhard Reiter wrote: > Didrik, > > this crash is strange, if you happen to have a Thuban on Windows, > could you try just to open the hs1-1500.shp file with it? > (Please write feedback into the tracker on wald, thanks!) > > Bernhard > > On Sunday 30 December 2007 19:32, thuban-bugs at wald.intevation.org wrote: > > Traceback (most recent call last): > > File "/usr/lib/thuban/Thuban/UI/view.py", line 190, in OnIdle > > event.RequestMore(self._do_redraw()) > > File "/usr/lib/python2.4/site-packages/wx-2.6-gtk2-unicode/wx/_core.py", > > line 6749, in RequestMore return _core_.IdleEvent_RequestMore(*args, > > **kwargs) > > MemoryError: argument number 1: > > > > I am using Thuban version: Thuban Release Version 1.2.0 > > on my Debian Sid. The layer that nade it crash was file hs1-1500.shp from > > http://www.juntadeandalucia.es/obraspublicasytransportes/www/estaticas/cart > >ografia/descargas/archivos/hidrografialineal.zip > > _______________________________________________ > Thuban-devel mailing list > Thuban-devel at intevation.de > https://intevation.de/mailman/listinfo/thuban-devel -- Didrik Pinte Information Technologies for the Agro-Environment -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080108/3bfaa24a/attachment.bin From bram.degreve at gmail.com Wed Jan 9 11:47:19 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Wed, 9 Jan 2008 11:47:19 +0100 Subject: WIP-pyshapelib-Unicode issues Message-ID: <1d5b08270801090247u2beb5321ieb529c4ad7c1189a@mail.gmail.com> Hi all, I've committed some changes to the unicode branch. I've fixed copyright infos and headers, the README, added an entry in NEWS for 0.4, and did some other work on the source code as well. The compiler warnings should be gone. If possible, can you guys test it to identify any further issues? You can use the WIP-pyshapelib-Unicode branch "out of the box", as no merge with the trunk is necessary (I did that for you). Known issues: * {test_load|test_load_1_0}.TestNonAsciiColumnName.test: the column name Fl?che is giving us troubles. Not in dbflib itself, as it is handled correctly over there, but apparently it is not encoded to UTF-8 when written to the session XML, causing the test to fail. I'm not familiar with this issue, so if anyone can help, please feel free =) * test_transientdb.TestTransientTable.{test_auto_transient_table|test_transient_table}: the shapelib C library returns integer numerical fields with more than 10 digits as double (in order to avoid an overflow with the C int). This is causing the test to fail as an 'int' is expected for column 3, and an 'double' is found. Bernard Herzog suggests to let pyshapelib convert them back to Python long integers. On the Python API level this can easily be done using PyLong_FromDouble. Is it OK to mix long integers and "short" integers, or should all integers coming from pyshapelib be converted to long ints? * when creating new DBF files, Thuban will use the LDID_ESRI_ANSI code page. That's LDID 0x57 and uses the cp1252 codec. This should be configurable by the user, but I don't really know where to start. Can anyone who's familier with the Thuban UI give a headstart? * pyshapelib should get a proper unittest that can run on its own, but is also tested from test/runtests.py. I've never made a unittest in Python before, so I'm a bit puzzled here. -- hi, i'm a signature viruz, plz set me as your signature and help me spread :) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.intevation.de/pipermail/thuban-devel/attachments/20080109/49dfb50f/attachment.html From bernhard at intevation.de Wed Jan 9 18:37:18 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Wed, 9 Jan 2008 18:37:18 +0100 Subject: WIP-pyshapelib-Unicode issues In-Reply-To: <1d5b08270801090247u2beb5321ieb529c4ad7c1189a@mail.gmail.com> References: <1d5b08270801090247u2beb5321ieb529c4ad7c1189a@mail.gmail.com> Message-ID: <200801091837.25954.bernhard@intevation.de> Hi Bram, On Wednesday 09 January 2008 11:47, Bram de Greve wrote: > I've committed some changes to the unicode branch. I've fixed copyright > infos and headers, the README, added an entry in NEWS for 0.4, and did some > other work on the source code as well. The compiler warnings should be > gone. cool, I saw some of the commits happening, but hunted the empty shape file problem. (Does somebody know, btw, if such a shape file is legitimate at all?) > * > test_transientdb.TestTransientTable.{test_auto_transient_table|test_transie >nt_table}: > > the shapelib C library returns integer numerical fields with more than 10 > digits as double (in order to avoid an overflow with the C int). This is > causing the test to fail as an 'int' is expected for column 3, and an > 'double' is found. Bernard Herzog suggests to let pyshapelib convert them > back to Python long integers. On the Python API level this can easily be > done using PyLong_FromDouble. Is it OK to mix long integers and "short" > integers, or should all integers coming from pyshapelib be converted to > long ints? Probably one class of objects should always have the same representation, so if "long" is sufficient for ids, so why not use it for all ids? This seems consistent. If there are values where (int *) is sufficient, we could use this. > * when creating new DBF files, Thuban will use the LDID_ESRI_ANSI code > page. That's LDID 0x57 and uses the cp1252 codec. Is this the default encoding old shapefiles uses to have? > This should be > configurable by the user, but I don't really know where to start. Can > anyone who's familier with the Thuban UI give a headstart? First we have to decide where to save this property. It looks like the property of a .dbf table which can be a table on its own or a part of a shapefile layer. If Thuban displays a table it will already have a concept about it's encoding. Otherwise it would need to recode it. So for files where Thuban cannot determine the encoding, we probably have to add something like in the "import" statements. Maybe the table view should also display this property. > * pyshapelib should get a proper unittest that can run on its own, but is > also tested from test/runtests.py. I've never made a unittest in Python > before, so I'm a bit puzzled here. Check the files in there, they are pretty good examples. It is more straightforward then you might think. :) -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080109/4414a8a6/attachment.bin From bram.degreve at gmail.com Wed Jan 9 18:52:15 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Wed, 09 Jan 2008 18:52:15 +0100 Subject: WIP-pyshapelib-Unicode issues In-Reply-To: <200801091837.25954.bernhard@intevation.de> References: <1d5b08270801090247u2beb5321ieb529c4ad7c1189a@mail.gmail.com> <200801091837.25954.bernhard@intevation.de> Message-ID: <478509CF.1020305@gmail.com> Hi Bernhard, Bernhard Reiter wrote: >> * >> test_transientdb.TestTransientTable.{test_auto_transient_table|test_transie >> nt_table}: >> >> the shapelib C library returns integer numerical fields with more than 10 >> digits as double (in order to avoid an overflow with the C int). This is >> causing the test to fail as an 'int' is expected for column 3, and an >> 'double' is found. Bernard Herzog suggests to let pyshapelib convert them >> back to Python long integers. On the Python API level this can easily be >> done using PyLong_FromDouble. Is it OK to mix long integers and "short" >> integers, or should all integers coming from pyshapelib be converted to >> long ints? >> > > Probably one class of objects should always have the same representation, > so if "long" is sufficient for ids, so why not use it for all ids? > This seems consistent. If there are values where (int *) is sufficient, > we could use this. > Sounds reasonable. >> * when creating new DBF files, Thuban will use the LDID_ESRI_ANSI code >> page. That's LDID 0x57 and uses the cp1252 codec. >> > > Is this the default encoding old shapefiles uses to have? > Yes ... and no. Probably, most shapefiles created with a default ESRI installation will have this encoding. But shapefiles created with the old dblfib did not have any associated code page at all! At any rate, dbflib totally ignored any code page when reading dbf files, so things pretty much went down to the default Python encoding, which is ... ASCII? > >> This should be >> configurable by the user, but I don't really know where to start. Can >> anyone who's familier with the Thuban UI give a headstart? >> > > First we have to decide where to save this property. > Is there any "config" file for Thuban? If so, I would save it there. > It looks like the property of a .dbf table which can be a table on its own > or a part of a shapefile layer. > Correct. > If Thuban displays a table it will already have a concept about it's encoding. > Otherwise it would need to recode it. > So for files where Thuban cannot determine the encoding, we probably have to > add something like in the "import" statements. > Good point! I totally missed the issue of dbf files that do not have any code page associated. Until now, I simply assumed cp1252. But that's not the only reason the have this encoding configurable. More important is the encoding to be used when _creating_ new dbf files! > Maybe the table view should also display this property. > That would be nice, though not really necessary. > >> * pyshapelib should get a proper unittest that can run on its own, but is >> also tested from test/runtests.py. I've never made a unittest in Python >> before, so I'm a bit puzzled here. >> > > Check the files in there, they are pretty good examples. It is more > straightforward then you might think. :) > I'll give it another try =) Bram From bernhard at intevation.de Wed Jan 9 19:50:45 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Wed, 9 Jan 2008 19:50:45 +0100 Subject: default dbflib encoding (Re: WIP-pyshapelib-Unicode issues) In-Reply-To: <478509CF.1020305@gmail.com> References: <1d5b08270801090247u2beb5321ieb529c4ad7c1189a@mail.gmail.com> <200801091837.25954.bernhard@intevation.de> <478509CF.1020305@gmail.com> Message-ID: <200801091950.45738.bernhard@intevation.de> On Wednesday 09 January 2008 18:52, Bram de Greve wrote: > so things > pretty much went down to the default Python encoding, which is ... ASCII? I think we should assume something that a majority of users used, like latin1 which is a superset of ascii. -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080109/86a1e438/attachment.bin From bernhard at intevation.de Wed Jan 9 19:54:55 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Wed, 9 Jan 2008 19:54:55 +0100 Subject: WIP-pyshapelib-Unicode issues In-Reply-To: <478509CF.1020305@gmail.com> References: <1d5b08270801090247u2beb5321ieb529c4ad7c1189a@mail.gmail.com> <200801091837.25954.bernhard@intevation.de> <478509CF.1020305@gmail.com> Message-ID: <200801091954.56041.bernhard@intevation.de> On Wednesday 09 January 2008 18:52, Bram de Greve wrote: > >> This should be > >> configurable by the user, but I don't really know where to start. ?Can > >> anyone who's familier with the Thuban UI give a headstart? > >> ? ? > > > > First we have to decide where to save this property. > > ? > > Is there any "config" file for Thuban? ?If so, I would save it there. This would be only suitable if it is a global option, but if I add a few shapefiles, they could come from different sources and have different encoding in the dbf files. Thus this looks like at least a property per layer. Thuban has a ~/.thubn/thubanstart.py file and the sessions files usually contain all the information necessary to reload a set of layers. > > It looks like the property of a .dbf table which can be a table on its > > own or a part of a shapefile layer. > > ? > Correct. > > > If Thuban displays a table it will already have a concept about it's > > encoding. Otherwise it would need to recode it. > > So for files where Thuban cannot determine the encoding, we probably have > > to add something like in the "import" statements. > > ? > > Good point! I totally missed the issue of dbf files that do not have any > code page associated. ?Until now, I simply assumed cp1252. > But that's not the only reason the have this encoding configurable. > More important is the encoding to be used when _creating_ new dbf files! Yes, this would be the export possibilities from the table view dialog. > > Maybe the table view should also display this property. > > That would be nice, though not really necessary. If it is not in the shapefile and cannot be reliably detected means the user must change it and it needs to be saved in the session file so it comes up again as the user wanted it to be. I think adding a display and button to the table view is a good approach. Bernhard -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080109/8dc050f0/attachment.bin From bram.degreve at gmail.com Wed Jan 9 20:33:30 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Wed, 09 Jan 2008 20:33:30 +0100 Subject: WIP-pyshapelib-Unicode issues In-Reply-To: <200801091954.56041.bernhard@intevation.de> References: <1d5b08270801090247u2beb5321ieb529c4ad7c1189a@mail.gmail.com> <200801091837.25954.bernhard@intevation.de> <478509CF.1020305@gmail.com> <200801091954.56041.bernhard@intevation.de> Message-ID: <4785218A.9080700@gmail.com> Bernhard Reiter wrote: > On Wednesday 09 January 2008 18:52, Bram de Greve wrote: > >>>> This should be >>>> configurable by the user, but I don't really know where to start. Can >>>> anyone who's familier with the Thuban UI give a headstart? >>>> >>>> >>> First we have to decide where to save this property. >>> >>> >> Is there any "config" file for Thuban? If so, I would save it there. >> > > This would be only suitable if it is a global option, > but if I add a few shapefiles, they could come from different sources > and have different encoding in the dbf files. > Thus this looks like at least a property per layer. > TERMINOLOGY ISSUE. I've been using the term code page for whatever the DBF file thinks is the code page. That's either an LDID constant or the content of the .cpg file. This is however not really down to the real meaning of code page. e.g. many of the LDID values refer to the same code page. e.g. LDID_DUTCH_OEM and LDID_FRENCH_OEM use the same "code page" cp437. OK, so the import dialog should have something like an checkable option to override the encoding. dbflib.open() and dbflib.DBFFile would need yet another optional argument to override the code page. *grasp* Actually, we don't need that. We can request dbflib to return encoded strings (default) and decode them ourselves, either via the file specific codec or via the overriding codec. And we would save this in the session. How do we name the encodings? If it is DBF specific, we can name them by their code page names (whatever follows the LDID_ and CPG_ constants). If it needs to be more general, we'll have to resort to Python codec names. You have to know that many of the code pages result in the same Python codec! > Thuban has a ~/.thubn/thubanstart.py file and the sessions files usually > contain all the information necessary to reload a set of layers. > > >> More important is the encoding to be used when _creating_ new dbf files! >> > > Yes, this would be the export possibilities from the table view dialog. > Ok, so selecting the code page when creating the dbf should be a non-optional selection box. However, I believe it would be useful if we store the user's preference in thubanstart.py. So that they don't have to select it over and over again if they use the same code page most of the time. > If it is not in the shapefile and cannot be reliably detected means the user > must change it and it needs to be saved in the session file so it comes up > again as the user wanted it to be. I think adding a display and button to the > table view is a good approach. > > You convinced me an overriden code page should be stored in the session file. And I won't say no to adding a display and a button either =) Bram From bernhard at intevation.de Thu Jan 10 11:56:16 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Thu, 10 Jan 2008 11:56:16 +0100 Subject: WIP-pyshapelib-Unicode issues In-Reply-To: <4785218A.9080700@gmail.com> References: <1d5b08270801090247u2beb5321ieb529c4ad7c1189a@mail.gmail.com> <200801091954.56041.bernhard@intevation.de> <4785218A.9080700@gmail.com> Message-ID: <200801101156.21760.bernhard@intevation.de> On Wednesday 09 January 2008 20:33, Bram de Greve wrote: > TERMINOLOGY ISSUE. ?I've been using the term code page for whatever the > DBF file thinks is the code page. ?That's either an LDID constant or the > content of the .cpg file. Maybe I do not know enough about how this all works. I am only trying to help you by asking questions and suggesting general things. My conception is that there are .dbf files (or group of files within a "shapefile") that we do not know the encoding of. For them we need the selection button, because Thuban cannot determine the encoding by itself. As for the default of what to create, I think we make it a logic coming out of the current language from the locale and what is most commonly created by other up-todate .shp and .dbf tools. The user should be able to override the default by setting a variable in thubanstart.py, I agree. Bernhard -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080110/f89d532a/attachment.bin From bram.degreve at gmail.com Fri Jan 11 18:44:53 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Fri, 11 Jan 2008 18:44:53 +0100 Subject: WIP-pyshapelib-Unicode issues In-Reply-To: <200801091837.25954.bernhard@intevation.de> References: <1d5b08270801090247u2beb5321ieb529c4ad7c1189a@mail.gmail.com> <200801091837.25954.bernhard@intevation.de> Message-ID: <4787AB15.9070201@gmail.com> Bernhard Reiter wrote: > Hi Bram, > > On Wednesday 09 January 2008 11:47, Bram de Greve wrote: > >> I've committed some changes to the unicode branch. I've fixed copyright >> infos and headers, the README, added an entry in NEWS for 0.4, and did some >> other work on the source code as well. The compiler warnings should be >> gone. >> > > cool, I saw some of the commits happening, but hunted the empty shape file > problem. (Does somebody know, btw, if such a shape file is legitimate at > all?) > > Why would it not be legitimate? I've skimmed the specs and found no such thing (about being illegal). >> * >> test_transientdb.TestTransientTable.{test_auto_transient_table|test_transie >> nt_table}: >> >> the shapelib C library returns integer numerical fields with more than 10 >> digits as double (in order to avoid an overflow with the C int). This is >> causing the test to fail as an 'int' is expected for column 3, and an >> 'double' is found. Bernard Herzog suggests to let pyshapelib convert them >> back to Python long integers. On the Python API level this can easily be >> done using PyLong_FromDouble. Is it OK to mix long integers and "short" >> integers, or should all integers coming from pyshapelib be converted to >> long ints? >> > > Probably one class of objects should always have the same representation, > so if "long" is sufficient for ids, so why not use it for all ids? > This seems consistent. If there are values where (int *) is sufficient, > we could use this. > > It turns out dbfopen.c reads all numbers as double anyway. Only if it things it's an integer and will fit in a C int (and thus be of type FTInteger), it casts the double to an int at the very last moment possible. I'm skipping that cast now and use PyLong_FromDouble to _always_ cast doubles to long integers iif the number of decimals is zero. That's two errors less in the unittests =) Cheers, Bramz From bernhard at intevation.de Mon Jan 14 08:51:05 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Mon, 14 Jan 2008 08:51:05 +0100 Subject: WIP-pyshapelib-Unicode issues In-Reply-To: <4787AB15.9070201@gmail.com> References: <1d5b08270801090247u2beb5321ieb529c4ad7c1189a@mail.gmail.com> <200801091837.25954.bernhard@intevation.de> <4787AB15.9070201@gmail.com> Message-ID: <200801140851.09875.bernhard@intevation.de> On Friday 11 January 2008 18:44, Bram de Greve wrote: > > cool, I saw some of the commits happening, but hunted the empty shape > > file problem. (Does somebody know, btw, if such a shape file is > > legitimate at all?) > Why would it not be legitimate? ? I don't know. Given Thuban's existance and that it is only reported now, makes me believe such shapefiles are rare. Also it would not really serve a purpose to have an object without geoobject in there. > I've skimmed the specs and found no > such thing (about being illegal). Okay, good to know! Thanks for looking into the specs. > It turns out dbfopen.c reads all numbers as double anyway. ?Only if it > things it's an integer and will fit in a C int (and thus be of type > FTInteger), it casts the double to an int at the very last moment > possible. ?I'm skipping that cast now and use PyLong_FromDouble to > _always_ cast doubles to long integers iif the number of decimals is > zero. ?That's two errors less in the unittests =) Sounds like this is the sensible way to go. Bernhard -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080114/cf7d6d5a/attachment.bin From thuban-bugs at wald.intevation.org Sun Jan 13 17:08:54 2008 From: thuban-bugs at wald.intevation.org (thuban-bugs@wald.intevation.org) Date: Sun, 13 Jan 2008 17:08:54 +0100 (CET) Subject: =?UTF-8?B?W3RodWJhbi1CdWdzXVs1NzddIG5vIGdlb3RpZmYgaW1hZ2UgbGF5ZXI=?= Message-ID: <20080113160854.B685840581@pyrosoma.intevation.org> Bugs item #577, was opened at 2008-01-13 16:08 Status: Open Priority: 3 Submitted By: Carlo van Rijswijk (hpb) Assigned to: Nobody (None) Summary: no geotiff image layer Resolution: None Version: None Category: None Initial Comment: Hi, Can somebody help me with the following? When I try to open an image layer (tiff or geotiff) in Thuban, I get the following message: An unhandled exception occurred global name 'filename' is not defined (please report to http://thuban.intevation.org/bugtracker.html) Traceback (most recent call last): File "thuban\buildthuban\out1.pyz/Thuban.UI.mainwindow", line 299, in invoke_command File "thuban\buildthuban\out1.pyz/Thuban.UI.command", line 121, in Execute File "thuban\buildthuban\out1.pyz/Thuban.UI.mainwindow", line 1071, in call_method File "thuban\buildthuban\out1.pyz/Thuban.UI.mainwindow", line 595, in AddRasterLayer NameError: global name 'filename' is not defined I' m using: Thuban 1.2 svn-19700101 on Ubuntu Gusty Gibbon trough the Wine emulator. with: wxPython 2.6.3.3 Python 2.4.4 PySQLite 2.3.3 SQLite 3.3.10 GDAL 1.5dev psycopg 2.0.5.1 (dec dt ext pq3) Internal encoding: cp1252 Thank you in advance, Carlo ---------------------------------------------------------------------- You can respond by visiting: http://wald.intevation.org/tracker/?func=detail&atid=105&aid=577&group_id=6 From bram.degreve at gmail.com Mon Jan 14 12:36:50 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Mon, 14 Jan 2008 12:36:50 +0100 Subject: WIP-pyshapelib-Unicode issues In-Reply-To: <200801140851.09875.bernhard@intevation.de> References: <1d5b08270801090247u2beb5321ieb529c4ad7c1189a@mail.gmail.com> <200801091837.25954.bernhard@intevation.de> <4787AB15.9070201@gmail.com> <200801140851.09875.bernhard@intevation.de> Message-ID: <1d5b08270801140336o69fd779akd15789c4b9e58be5@mail.gmail.com> On 14/01/2008, Bernhard Reiter wrote: > > On Friday 11 January 2008 18:44, Bram de Greve wrote: > > > cool, I saw some of the commits happening, but hunted the empty shape > > > file problem. (Does somebody know, btw, if such a shape file is > > > legitimate at all?) > > > Why would it not be legitimate? > > I don't know. Given Thuban's existance and that it is only reported now, > makes me believe such shapefiles are rare. Also it would not really serve > a > purpose to have an object without geoobject in there. > > > I've skimmed the specs and found no > > such thing (about being illegal). > > Okay, good to know! Thanks for looking into the specs. I've looked a bit further into it and I've found the conclusive paragraph in the specs on page 5: "If the shapefile is empty (that is, has no records), the values for Xmin, Ymin, Xmax and Ymax are unspecified." To me, that implies that they are valid. Also, there's something like null shapes, without geometric data. Each shapefile type supports null shapes, so you can have points and nulls in one file. But not points and lines ... Bramz -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.intevation.de/pipermail/thuban-devel/attachments/20080114/60e12b6b/attachment.html From bernhard at intevation.de Mon Jan 14 15:51:24 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Mon, 14 Jan 2008 15:51:24 +0100 Subject: WIP-pyshapelib-Unicode issues In-Reply-To: <1d5b08270801140336o69fd779akd15789c4b9e58be5@mail.gmail.com> References: <1d5b08270801090247u2beb5321ieb529c4ad7c1189a@mail.gmail.com> <200801140851.09875.bernhard@intevation.de> <1d5b08270801140336o69fd779akd15789c4b9e58be5@mail.gmail.com> Message-ID: <200801141551.28499.bernhard@intevation.de> On Monday 14 January 2008 12:36, Bram de Greve wrote: > I've looked a bit further into it and I've found the conclusive paragraph > in the specs on page 5: > > "If the shapefile is empty (that is, has no records), the values for Xmin, > Ymin, Xmax and Ymax are unspecified." Hmm there are reconrd in the shapefile, there is only one record without geometry. Check the link to an example shapefile in the reported issue. > To me, that implies that they are valid. > > Also, there's something like null shapes, without geometric data. ?Each > shapefile type supports null shapes, so you can have points and nulls in > one file. ?But not points and lines ... The above paragraph seems to be too short for me to fully understand. If it is okay to not have geographic data in a shapeobject, I guess the example shapefile would be legitimate. -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080114/6acd5812/attachment.bin From bram.degreve at gmail.com Mon Jan 14 16:19:23 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Mon, 14 Jan 2008 16:19:23 +0100 Subject: null shapes (was Re: WIP-pyshapelib-Unicode issues) In-Reply-To: <200801141551.28499.bernhard@intevation.de> References: <1d5b08270801090247u2beb5321ieb529c4ad7c1189a@mail.gmail.com> <200801140851.09875.bernhard@intevation.de> <1d5b08270801140336o69fd779akd15789c4b9e58be5@mail.gmail.com> <200801141551.28499.bernhard@intevation.de> Message-ID: <478B7D7B.7030808@gmail.com> Bernhard Reiter wrote: > On Monday 14 January 2008 12:36, Bram de Greve wrote: > >> I've looked a bit further into it and I've found the conclusive paragraph >> in the specs on page 5: >> >> "If the shapefile is empty (that is, has no records), the values for Xmin, >> Ymin, Xmax and Ymax are unspecified." >> > > Hmm there are reconrd in the shapefile, > there is only one record without geometry. > Check the link to an example shapefile in the reported issue. > > Oh, I was under the assumption you were talking about shapefiles _without_ records =) "cool, I saw some of the commits happening, but hunted the /empty shape file/ problem. (Does somebody know, btw, if such a shape file is legitimate at all?)" [emphasis is mine] >> To me, that implies that they are valid. >> >> Also, there's something like null shapes, without geometric data. Each >> shapefile type supports null shapes, so you can have points and nulls in >> one file. But not points and lines ... >> > > The above paragraph seems to be too short for me to fully understand. > If it is okay to not have geographic data in a shapeobject, I guess the > example shapefile would be legitimate. > At the bottom of page 5 of the specs (http://www.esri.com/library/whitepapers/pdfs/shapefile.pdf), it states: "A shape type of 0 indicates a null shape, with no geometric data for the shape. Each feature type (point, line, polygon, etc.) [this would be the type of the /file/, Bram] supports nulls - it is valid to have points and null points in the same shapefile." So, yes, the example shapefile is legitimate. There are actually 109 null shapes in that file (on 4962 shapes in total). Bram From bram.degreve at gmail.com Mon Jan 14 16:27:29 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Mon, 14 Jan 2008 16:27:29 +0100 Subject: default dbflib encoding (Re: WIP-pyshapelib-Unicode issues) In-Reply-To: <200801091950.45738.bernhard@intevation.de> References: <1d5b08270801090247u2beb5321ieb529c4ad7c1189a@mail.gmail.com> <200801091837.25954.bernhard@intevation.de> <478509CF.1020305@gmail.com> <200801091950.45738.bernhard@intevation.de> Message-ID: <478B7F61.3010302@gmail.com> Bernhard Reiter wrote: > On Wednesday 09 January 2008 18:52, Bram de Greve wrote: > >> so things >> pretty much went down to the default Python encoding, which is ... ASCII? >> > > I think we should assume something that a majority of users used, > like latin1 which is a superset of ascii. > > > Either latin1 (cp819) or cp1252 (ansi western europe) ... I believe ESRI tends more towards the latter, but as Thuban is more Linux oriented, it might be that the former is a better choice (in terms of what a majority of Thuban users used). Bram > ------------------------------------------------------------------------ > > _______________________________________________ > Thuban-devel mailing list > Thuban-devel at intevation.de > https://intevation.de/mailman/listinfo/thuban-devel > From bram.degreve at gmail.com Mon Jan 14 16:30:23 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Mon, 14 Jan 2008 16:30:23 +0100 Subject: WIP-pyshapelib-Unicode issues In-Reply-To: <200801101156.21760.bernhard@intevation.de> References: <1d5b08270801090247u2beb5321ieb529c4ad7c1189a@mail.gmail.com> <200801091954.56041.bernhard@intevation.de> <4785218A.9080700@gmail.com> <200801101156.21760.bernhard@intevation.de> Message-ID: <478B800F.6030700@gmail.com> Bernhard Reiter wrote: > Maybe I do not know enough about how this all works. > I am only trying to help you by asking questions and suggesting general > things. > > My conception is that there are .dbf files (or group of files within a > "shapefile") that we do not know the encoding of. > For them we need the selection button, > because Thuban cannot determine the encoding by itself. > > As for the default of what to create, I think we make it a logic > coming out of the current language from the locale and what is most commonly > created by other up-todate .shp and .dbf tools. > The user should be able to override the default by setting a variable > in thubanstart.py, I agree. > > I can agree with this. Bram From thuban-bugs at wald.intevation.org Mon Jan 14 19:54:53 2008 From: thuban-bugs at wald.intevation.org (thuban-bugs@wald.intevation.org) Date: Mon, 14 Jan 2008 19:54:53 +0100 (CET) Subject: =?UTF-8?B?W3RodWJhbi1CdWdzXVs1NzhdIEJ1aWxkaW5nIGFuIHVwZGF0ZWQgLmRlYiBwYWNrYWdl?= Message-ID: <20080114185453.4C57E406C5@pyrosoma.intevation.org> Bugs item #578, was opened at 2008-01-14 18:54 Status: Open Priority: 3 Submitted By: Carlo van Rijswijk (hpb) Assigned to: Nobody (None) Summary: Building an updated .deb package Resolution: None Version: None Category: None Initial Comment: Hi, I'm using Thuban under Ubuntu, Gutsy Gibbon. I would like to use a newer version though, but I have no skills to build it (1.2.0 - with extensions as svg-export) from source. So I would like to ask if someone has (or could build??) a .deb package for me. Thanks, Carlo ---------------------------------------------------------------------- You can respond by visiting: http://wald.intevation.org/tracker/?func=detail&atid=105&aid=578&group_id=6 From bernhard at intevation.de Tue Jan 15 15:17:09 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Tue, 15 Jan 2008 15:17:09 +0100 Subject: null shapes (was Re: WIP-pyshapelib-Unicode issues) In-Reply-To: <478B7D7B.7030808@gmail.com> References: <1d5b08270801090247u2beb5321ieb529c4ad7c1189a@mail.gmail.com> <200801141551.28499.bernhard@intevation.de> <478B7D7B.7030808@gmail.com> Message-ID: <200801151517.15183.bernhard@intevation.de> On Monday 14 January 2008 16:19, Bram de Greve wrote: > Oh, I was under the assumption you were talking about shapefiles > _without_ records =) > > "cool, I saw some of the commits happening, but hunted the /empty shape > file/ problem. (Does somebody know, btw, if such a shape file is > legitimate at all?)" [emphasis is mine] Ah, sorry for the confusion. At least we've got it clarified. > At the bottom of page 5 of the specs > (http://www.esri.com/library/whitepapers/pdfs/shapefile.pdf), it states: > > "A shape type of 0 indicates a null shape, with no geometric data for > the shape. ?Each feature type (point, line, polygon, etc.) [this would > be the type of the /file/, Bram] supports nulls - it is valid to have > points and null points in the same shapefile." > > So, yes, the example shapefile is legitimate. There are actually 109 > null shapes in that file (on 4962 shapes in total). Very good to know! Thanks a lot for looking this up! I guess we should produce an example shapefile having one null element and one with points and add a unittest based on this file. Is it easy to create such a shapefile with shapelib? -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080115/2aa6f5bf/attachment.bin From bram.degreve at gmail.com Tue Jan 15 15:41:39 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Tue, 15 Jan 2008 15:41:39 +0100 Subject: null shapes (was Re: WIP-pyshapelib-Unicode issues) In-Reply-To: <200801151517.15183.bernhard@intevation.de> References: <1d5b08270801090247u2beb5321ieb529c4ad7c1189a@mail.gmail.com> <200801141551.28499.bernhard@intevation.de> <478B7D7B.7030808@gmail.com> <200801151517.15183.bernhard@intevation.de> Message-ID: <478CC623.1060708@gmail.com> Bernhard Reiter wrote: > On Monday 14 January 2008 16:19, Bram de Greve wrote: > >> Oh, I was under the assumption you were talking about shapefiles >> _without_ records =) >> >> "cool, I saw some of the commits happening, but hunted the /empty shape >> file/ problem. (Does somebody know, btw, if such a shape file is >> legitimate at all?)" [emphasis is mine] >> > > Ah, sorry for the confusion. At least we've got it clarified. > > >> At the bottom of page 5 of the specs >> (http://www.esri.com/library/whitepapers/pdfs/shapefile.pdf), it states: >> >> "A shape type of 0 indicates a null shape, with no geometric data for >> the shape. Each feature type (point, line, polygon, etc.) [this would >> be the type of the /file/, Bram] supports nulls - it is valid to have >> points and null points in the same shapefile." >> >> So, yes, the example shapefile is legitimate. There are actually 109 >> null shapes in that file (on 4962 shapes in total). >> > > Very good to know! Thanks a lot for looking this up! > I guess we should produce an example shapefile having one null element > and one with points and add a unittest based on this file. > Is it easy to create such a shapefile with shapelib? > I would guess so. I've skimmed over the source code, and it should give no problems at all. Just create a null shape as following: obj = shapelib.SHPObject(shapelib.SHPT_NULL, -1, []) and add it to the shapefile. Bramz > > ------------------------------------------------------------------------ > > _______________________________________________ > Thuban-devel mailing list > Thuban-devel at intevation.de > https://intevation.de/mailman/listinfo/thuban-devel > From bram.degreve at gmail.com Wed Jan 16 13:52:33 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Wed, 16 Jan 2008 13:52:33 +0100 Subject: pyshapelib: python requirements Message-ID: <478DFE11.4040506@gmail.com> Hi, I've tried to build pyshapelib on linux today (as I have access to it again), and it broke on PyOS_ascii_atof as it is new in Python 2.4 and the linux box is still running 2.3.5. What do we do? Bump up the requirements or compile conditionally? What do we know about the atof situation prior to Python 2.4? Cheers, Bram From bernhard at intevation.de Wed Jan 16 19:22:46 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Wed, 16 Jan 2008 19:22:46 +0100 Subject: pyshapelib: python requirements In-Reply-To: <478DFE11.4040506@gmail.com> References: <478DFE11.4040506@gmail.com> Message-ID: <200801161922.47571.bernhard@intevation.de> On Wednesday 16 January 2008 13:52, Bram de Greve wrote: > What do we do? ?Bump up the requirements or compile conditionally? > > What do we know about the atof situation prior to Python 2.4? I think my old code had provisions or explanations for this. -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080116/38ec8307/attachment.bin From bram.degreve at gmail.com Wed Jan 16 22:11:36 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Wed, 16 Jan 2008 22:11:36 +0100 Subject: pyshapelib: python requirements In-Reply-To: <200801161922.47571.bernhard@intevation.de> References: <478DFE11.4040506@gmail.com> <200801161922.47571.bernhard@intevation.de> Message-ID: <478E7308.3060704@gmail.com> Bernhard Reiter wrote: > On Wednesday 16 January 2008 13:52, Bram de Greve wrote: > >> What do we do? Bump up the requirements or compile conditionally? >> >> What do we know about the atof situation prior to Python 2.4? >> > > I think my old code had provisions or explanations for this. > > Well, in your old code I've found that you conditionally used PyOS_ascii_atof based on the Python version. That's what I do now as well. Not sure if it is a good idea though, as it will show different behaviour when compiled with Python < 2.4. From bernhard at intevation.de Thu Jan 17 10:54:01 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Thu, 17 Jan 2008 10:54:01 +0100 Subject: pyshapelib: python requirements In-Reply-To: <478E7308.3060704@gmail.com> References: <478DFE11.4040506@gmail.com> <200801161922.47571.bernhard@intevation.de> <478E7308.3060704@gmail.com> Message-ID: <200801171054.13631.bernhard@intevation.de> On Wednesday 16 January 2008 22:11, Bram de Greve wrote: > Well, in your old code I've found that you conditionally used > PyOS_ascii_atof based on the Python version. ?That's what I do now as well. > Not sure if it is a good idea though, as it will show different > behaviour when compiled with Python < 2.4. I thought I had comments in there or in the documentation. Darkly I remember that Python <2.4 will not set the LC_NUMERIC when calling the extention modules, so there is no need for the PyOS_ascii_atof() function and also no problem. So the behaviour should be the same, an LC_NUMERIC agnostic atof() is called. -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080117/a87f22ca/attachment.bin From bernhard at intevation.de Sat Jan 19 02:27:32 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Sat, 19 Jan 2008 02:27:32 +0100 Subject: Translations for 1.2.1 needed. Message-ID: <200801190227.36357.bernhard@intevation.de> While doing steps towards a 1.2.1 release I have noticed that the Extensions were not localised yet. They had not been added as POT source file, though the files itself already use the gettext funcation. Now I have changed this. Subsequently there are a lot new strings in the po files. Status as of Revision 2814: de.po: 416 translated messages, 62 fuzzy translations, 91 untranslated es.po: 410 translated messages, 65 fuzzy translations, 94 untranslated fr.po: 410 translated messages, 65 fuzzy translations, 94 untranslated hu.po: 384 translated messages, 78 fuzzy translations, 107 untranslated it.po: 356 translated messages, 94 fuzzy translations, 119 untranslated pt_BR.po: 362 translated messages, 88 fuzzy translations, 119 untranslated ru.po: 351 translated messages, 99 fuzzy translations, 119 untranslated Translation updates are appreciated of course. I will not delay 1.2.1, but it might be a few days until I get to make the final steps of the release. My workschedule is to first check for return values from wx that need conversion to the internal encoding, then to the NEWS and Releasenotes. Best, Bernhard -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080119/e1db9361/attachment.bin From dpinte at itae.be Mon Jan 21 11:15:11 2008 From: dpinte at itae.be (Didrik Pinte) Date: Mon, 21 Jan 2008 11:15:11 +0100 Subject: Translations for 1.2.1 needed. In-Reply-To: <200801190227.36357.bernhard@intevation.de> References: <200801190227.36357.bernhard@intevation.de> Message-ID: <1200910511.4651.0.camel@ddp.simpson> On Sat, 2008-01-19 at 02:27 +0100, Bernhard Reiter wrote: > Translation updates are appreciated of course. I've updated all the missing french translations. It's commited in the SVN (rev 2815). Didrik -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080121/bfcc668b/attachment.bin From bernhard at intevation.de Tue Jan 22 10:12:50 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Tue, 22 Jan 2008 10:12:50 +0100 Subject: Translations for 1.2.1 needed. In-Reply-To: <1200910511.4651.0.camel@ddp.simpson> References: <200801190227.36357.bernhard@intevation.de> <1200910511.4651.0.camel@ddp.simpson> Message-ID: <200801221012.56337.bernhard@intevation.de> On Monday 21 January 2008 11:15, Didrik Pinte wrote: > I've updated all the missing french translations. It's commited in the > SVN (rev 2815). Wonderful! :) Now French is better than German. ;) -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080122/78166a2e/attachment.bin From dpinte at itae.be Fri Jan 25 09:34:01 2008 From: dpinte at itae.be (Didrik Pinte) Date: Fri, 25 Jan 2008 09:34:01 +0100 Subject: AW: AW: [Thuban-list] Problems with changing the ProjectioninThuban Sessions In-Reply-To: <20080124212200.0EB1B1E0949@mx2.messagingengine.com> References: <20080124212200.0EB1B1E0949@mx2.messagingengine.com> Message-ID: <1201250041.5788.46.camel@ddp.simpson> On Thu, 2008-01-24 at 22:20 +0100, Stefanie wrote: > Yes I run the Windows Version and I installed the Programme just 2 months > (or so) ago! So I think it is the newest Version. > Stefanie, You have discovered a bug in Thuban. I've reported the problem over here : http://wald.intevation.org/tracker/index.php?func=detail&aid=586&group_id=6&atid=105 We will try to correct this as soon as possible. Probably for version 1.2.1 that must be released soon ;-) Didrik From dpinte at itae.be Fri Jan 25 16:16:05 2008 From: dpinte at itae.be (Didrik Pinte) Date: Fri, 25 Jan 2008 16:16:05 +0100 Subject: pyprojection problem Message-ID: <1201274165.5788.63.camel@ddp.simpson> Hi guys, It seems there is a problem with input arguments using pyprojection. When calling Projectionc.new_Projection(*args) with some values as unicode and not str, it crashes ! I guess it's a problem with the swig interface but I really don't know how to solve it. Any hints ? Didrik From bernhard at intevation.de Fri Jan 25 16:33:33 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Fri, 25 Jan 2008 16:33:33 +0100 Subject: pyprojection problem In-Reply-To: <1201274165.5788.63.camel@ddp.simpson> References: <1201274165.5788.63.camel@ddp.simpson> Message-ID: <200801251633.33953.bernhard@intevation.de> Didrik, On Friday 25 January 2008 16:16, Didrik Pinte wrote: > It seems there is a problem with input arguments using pyprojection. Is this [#586] projection exception? I could not reproduce this problem with current svn trunk. Can you? If so, please add the steps that reproduce it for you in the issue. > When calling Projectionc.new_Projection(*args) with some values as > unicode and not str, it crashes ! Internal encoding usually is str, but could be utf-8. So I do not thing this should be called with unicode strings. > I guess it's a problem with the swig interface but I really don't know > how to solve it. Can you give me a way to reproduce this? (Best is a testcase that we can later use.) :) Bernhard -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080125/c41c5f62/attachment.bin From bram.degreve at gmail.com Fri Jan 25 16:35:22 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Fri, 25 Jan 2008 16:35:22 +0100 Subject: pyprojection problem In-Reply-To: <1201274165.5788.63.camel@ddp.simpson> References: <1201274165.5788.63.camel@ddp.simpson> Message-ID: <479A01BA.1020106@gmail.com> Didrik Pinte wrote: > Hi guys, > > It seems there is a problem with input arguments using pyprojection. > > When calling Projectionc.new_Projection(*args) with some values as > unicode and not str, it crashes ! > > I guess it's a problem with the swig interface but I really don't know > how to solve it. > > Any hints ? > > Didrik > By crafting your own, like pyshapelib? =) /me ducks Seriously now, I'm took a _very_ quick glance into projection_wrap.c and pyprojection.i and the culprit seems to be lines 65 to 68 in projection.i. You recognize the error message on line 68? =) Two different angles of attack: (a) modify projection.i to also accept unicode objects, and hope you find a swig willing to compile projection.i (b) convert unicode strings to regular strings _in_Python_ before creating a Projection object. Remember that you need to encode the unicode string into a narrow character string at some point. So you'll have to pick an encoding. Either ASCII and throw exceptions when non-ASCII characters are found (or ANSI or whatever). Not a big problem as any encoding function will throw that exception for you. Or encode as UTF-8 ... Cheers, Bramz From bram.degreve at gmail.com Fri Jan 25 16:39:24 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Fri, 25 Jan 2008 16:39:24 +0100 Subject: pyprojection problem In-Reply-To: <479A01BA.1020106@gmail.com> References: <1201274165.5788.63.camel@ddp.simpson> <479A01BA.1020106@gmail.com> Message-ID: <479A02AC.7020302@gmail.com> Bram de Greve wrote: > Remember that you need to encode the unicode string into a narrow > character string at some point. So you'll have to pick an encoding. > Either ASCII and throw exceptions when non-ASCII characters are found > (or ANSI or whatever). Not a big problem as any encoding function will > throw that exception for you. Or encode as UTF-8 ... > Of course, you should first investigate if there's any particular encoding the PROJ.4 library is expecting. I presume that will not be UTF-8 =) > Cheers, > Bramz > > From dpinte at itae.be Fri Jan 25 16:48:18 2008 From: dpinte at itae.be (Didrik Pinte) Date: Fri, 25 Jan 2008 16:48:18 +0100 Subject: pyprojection problem In-Reply-To: <200801251633.33953.bernhard@intevation.de> References: <1201274165.5788.63.camel@ddp.simpson> <200801251633.33953.bernhard@intevation.de> Message-ID: <1201276098.5788.79.camel@ddp.simpson> On Fri, 2008-01-25 at 16:33 +0100, Bernhard Reiter wrote: > Didrik, > > On Friday 25 January 2008 16:16, Didrik Pinte wrote: > > It seems there is a problem with input arguments using pyprojection. > > Is this [#586] projection exception? Yes it's [#586]. > I could not reproduce this problem with current svn trunk. > Can you? If so, please add the steps that reproduce it for you in the issue. I'm running svn version 2815 (so latest trunk) version. Steps to reproduce : 1. launch thuban 2. Open session -> load Data/icelan_sample.thuban 3. Select political layer projection menu by right clicking on political layer 4. choose "Gauss-Krueger Zone 2 (Germany)" projection 5. click on button Try --> it should crach I've just seen that using "Geographic" projection did not crash. It seems the problem is related to text boxes (Latitude, longitude, etc.) that produce a unicode output. Can you reproduce it ? I can reproduce it with Debian 1.2.0-2 version, trunk version on Debian system, and Stefanie can do it on windows ;-) Didrik From dpinte at itae.be Fri Jan 25 16:54:21 2008 From: dpinte at itae.be (Didrik Pinte) Date: Fri, 25 Jan 2008 16:54:21 +0100 Subject: pyprojection problem In-Reply-To: <479A02AC.7020302@gmail.com> References: <1201274165.5788.63.camel@ddp.simpson> <479A01BA.1020106@gmail.com> <479A02AC.7020302@gmail.com> Message-ID: <1201276461.5788.83.camel@ddp.simpson> On Fri, 2008-01-25 at 16:39 +0100, Bram de Greve wrote: > Bram de Greve wrote: > > Remember that you need to encode the unicode string into a narrow > > character string at some point. So you'll have to pick an encoding. > > Either ASCII and throw exceptions when non-ASCII characters are found > > (or ANSI or whatever). Not a big problem as any encoding function will > > throw that exception for you. Or encode as UTF-8 ... > > > Of course, you should first investigate if there's any particular > encoding the PROJ.4 library is expecting. I presume that will not be > UTF-8 =) > > Cheers, > > Bramz You're right. We don't need any encoding over there. Problem seems to be output of wx text boxes that are unicode strings. If this is changed, everything will be ok ;-) Didrik From dpinte at itae.be Fri Jan 25 16:58:41 2008 From: dpinte at itae.be (Didrik Pinte) Date: Fri, 25 Jan 2008 16:58:41 +0100 Subject: pyprojection problem In-Reply-To: <1201276098.5788.79.camel@ddp.simpson> References: <1201274165.5788.63.camel@ddp.simpson> <200801251633.33953.bernhard@intevation.de> <1201276098.5788.79.camel@ddp.simpson> Message-ID: <1201276721.5788.88.camel@ddp.simpson> On Fri, 2008-01-25 at 16:48 +0100, Didrik Pinte wrote: > On Fri, 2008-01-25 at 16:33 +0100, Bernhard Reiter wrote: > > Didrik, > > > > On Friday 25 January 2008 16:16, Didrik Pinte wrote: > > > It seems there is a problem with input arguments using pyprojection. > > > > Is this [#586] projection exception? And here is a workaround/patch : Line 789 and following of Thuban/UI/projdialog.py params = ["proj=tmerc", "lat_0=" + str(self.__latitude.GetValue()), "lon_0=" + str(self.__longitude.GetValue()), "x_0=" + str(self.__falseEast.GetValue()), "y_0=" + str(self.__falseNorth.GetValue()), "k=" + str(self.__scale.GetValue())] params.extend(ProjPanel.GetParameters(self)) return params Adding a str() to each access to parameters solved the problem ... Is this acceptable ? There is no way to have strange encoding problem with those parameters (lat_0, etc.). Didrik From bram.degreve at gmail.com Fri Jan 25 17:03:25 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Fri, 25 Jan 2008 17:03:25 +0100 Subject: pyprojection problem In-Reply-To: <1201276721.5788.88.camel@ddp.simpson> References: <1201274165.5788.63.camel@ddp.simpson> <200801251633.33953.bernhard@intevation.de> <1201276098.5788.79.camel@ddp.simpson> <1201276721.5788.88.camel@ddp.simpson> Message-ID: <479A084D.9090604@gmail.com> Didrik Pinte wrote: > And here is a workaround/patch : > > Line 789 and following of Thuban/UI/projdialog.py > > params = ["proj=tmerc", > "lat_0=" + str(self.__latitude.GetValue()), > "lon_0=" + str(self.__longitude.GetValue()), > "x_0=" + str(self.__falseEast.GetValue()), > "y_0=" + str(self.__falseNorth.GetValue()), > "k=" + str(self.__scale.GetValue())] > params.extend(ProjPanel.GetParameters(self)) > return params > > Adding a str() to each access to parameters solved the problem ... > > Is this acceptable ? There is no way to have strange encoding problem > with those parameters (lat_0, etc.). > ProjPanel.GetParameters(self) can't do any funny things? I would suggest to move over the patch to Model/proj.py, as that seems to be a wrapper around pyprojection. I believe we're OK when you replace line 78 by: BaseProjection.__init__(self, map(str, params)) NOT TESTED ;) > Didrik > > _______________________________________________ > Thuban-devel mailing list > Thuban-devel at intevation.de > https://intevation.de/mailman/listinfo/thuban-devel > From bernhard at intevation.de Sun Jan 27 23:16:39 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Sun, 27 Jan 2008 23:16:39 +0100 Subject: pyprojection problem In-Reply-To: <479A084D.9090604@gmail.com> References: <1201274165.5788.63.camel@ddp.simpson> <1201276721.5788.88.camel@ddp.simpson> <479A084D.9090604@gmail.com> Message-ID: <200801272316.46143.bernhard@intevation.de> On Friday 25 January 2008 17:03, Bram de Greve wrote: > > Adding a str() to each access to parameters solved the problem ... > > > > Is this acceptable ? There is no way to have strange encoding problem > > with those parameters (lat_0, etc.). > > ? > > ProjPanel.GetParameters(self) can't do any funny things? > > I would suggest to move over the patch to Model/proj.py, as that seems > to be a wrapper around pyprojection. > I believe we're OK when you replace line 78 by: > > ? ? ? ? BaseProjection.__init__(self, map(str, params)) > > NOT TESTED ;) Thanks for your suggestions, I think I have solved this (see the issue for details). In general I believe the tactic should be to have a working 1.2.1 where we can and then port all internal representation to unicode objects. Then of course we must make sure to give the libraries what they want, I guess it will be ascii in the case of proj. Bernhard -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080127/94d4ec2c/attachment.bin From thuban-bugs at wald.intevation.org Fri Jan 25 09:29:59 2008 From: thuban-bugs at wald.intevation.org (thuban-bugs@wald.intevation.org) Date: Fri, 25 Jan 2008 09:29:59 +0100 (CET) Subject: =?UTF-8?B?W3RodWJhbi1CdWdzXVs1ODZdIHByb2plY3Rpb24gZXhjZXB0aW9u?= Message-ID: <20080125082959.8BBE740550@pyrosoma.intevation.org> Bugs item #586, was opened at 2008-01-25 09:29 Status: Open Priority: 3 Submitted By: Didrik Pinte (dpinte) Assigned to: Nobody (None) Summary: projection exception Resolution: None Version: None Category: None Initial Comment: Hi, Thanks to Stefanie , the following bug has been discovered. When trying to apply a projection to a loaded layer, the following exception occur : An unhandled exception occurred: list must contain strings (please report to http://thuban.intevation.org/bugtracker.html) Traceback (most recent call last): File "/usr/lib/thuban/Thuban/UI/projdialog.py", line 233, in OnApply self.__SetProjection() File "/usr/lib/thuban/Thuban/UI/projdialog.py", line 555, in __SetProjection self.receiver.SetProjection(self.__GetProjection()) File "/usr/lib/thuban/Thuban/UI/projdialog.py", line 578, in __GetProjection return Projection(parameters, self.projname.GetValue()) File "/usr/lib/thuban/Thuban/Model/proj.py", line 80, in __init__ BaseProjection.__init__(self, params) File "/usr/lib/thuban/Lib/Projection.py", line 5, in __init__ self.this = apply(Projectionc.new_Projection,args) TypeError: list must contain strings It seems to happen under all the 1.2.0 version (debian and windows). ---------------------------------------------------------------------- You can respond by visiting: http://wald.intevation.org/tracker/?func=detail&atid=105&aid=586&group_id=6 From bram.degreve at gmail.com Tue Jan 29 20:14:12 2008 From: bram.degreve at gmail.com (Bram de Greve) Date: Tue, 29 Jan 2008 20:14:12 +0100 Subject: WIP-pyshapelib-Bramz Message-ID: <479F7B04.9090905@gmail.com> Hi, I'm going to remove the WIP-pyshapelib-Bramz branch as it is not being developed anymore anyway, and I think we can better focus on the WIP-pyshapelib-Unicode branch. BTW: about GUI changes in Thuban to allow to override or choose shapelib encodings, I unfortunately cannot spare any time to do that. So if anyone wants to contribute ... Also, there's still that test error in test_load_1_0.py that I don't know how to solve. Bramz From bernhard at intevation.de Tue Jan 29 23:45:56 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Tue, 29 Jan 2008 23:45:56 +0100 Subject: WIP-pyshapelib-Bramz In-Reply-To: <479F7B04.9090905@gmail.com> References: <479F7B04.9090905@gmail.com> Message-ID: <200801292345.57309.bernhard@intevation.de> On Tuesday 29 January 2008 20:14, Bram de Greve wrote: > I'm going to remove the WIP-pyshapelib-Bramz branch as it is not being > developed anymore anyway, and I think we can better focus on the > WIP-pyshapelib-Unicode branch. Fine with me. svn will keep it as archived I believe so there is no loss. > BTW: about GUI changes in Thuban to allow to override or choose shapelib > encodings, I unfortunately cannot spare any time to do that. ?So if > anyone wants to contribute ... ?Also, there's still that test error in > test_load_1_0.py that I don't know how to solve. First I need to get the 1.2.1 release out of the door, then I will try to help with the WIP-pyshapelib-Unicode branch. -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080129/3376672d/attachment.bin From bernhard at intevation.de Wed Jan 30 11:29:00 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Wed, 30 Jan 2008 11:29:00 +0100 Subject: 1.2.1rc1 = svn2822 Message-ID: <200801301129.01533.bernhard@intevation.de> Hi Thuban-Developers or Friends, Revision: 2822 of trunk is the 1.2.1 release candidate. Didrik and I are doing some last tests and will release immedeately after. So if you want to give it a spin to find some show stoppers, this is your change. :) Bernhard -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080130/3b92651f/attachment.bin From bernhard at intevation.de Thu Jan 31 14:33:34 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Thu, 31 Jan 2008 14:33:34 +0100 Subject: 1.2.1rc1 = svn2824 In-Reply-To: <200801301129.01533.bernhard@intevation.de> References: <200801301129.01533.bernhard@intevation.de> Message-ID: <200801311433.38358.bernhard@intevation.de> On Wednesday 30 January 2008 11:29, Bernhard Reiter wrote: > Revision: 2822 of trunk is the 1.2.1 release candidate. Okay it is 2824 now. One translation added, thanks to Jachym. > Didrik and I are doing some last tests and will release immedeately after. > So if you want to give it a spin to find some show stoppers, this is your > change. :) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080131/b193ea47/attachment.bin From bernhard at intevation.de Thu Jan 31 14:59:06 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Thu, 31 Jan 2008 14:59:06 +0100 Subject: 1.2.1rc1 = svn2825 In-Reply-To: <200801311433.38358.bernhard@intevation.de> References: <200801301129.01533.bernhard@intevation.de> <200801311433.38358.bernhard@intevation.de> Message-ID: <200801311459.13229.bernhard@intevation.de> On Thursday 31 January 2008 14:33, Bernhard Reiter wrote: > On Wednesday 30 January 2008 11:29, Bernhard Reiter wrote: > > Revision: 2822 of trunk is the 1.2.1 release candidate. > > Okay it is 2824 now. > One translation added, thanks to Jachym. Now 2825, sorry for the noise. -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080131/c577e124/attachment.bin From bernhard at intevation.de Thu Jan 31 16:06:41 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Thu, 31 Jan 2008 16:06:41 +0100 Subject: 1.2.1rc1 = svn2825 In-Reply-To: <200801311459.13229.bernhard@intevation.de> References: <200801301129.01533.bernhard@intevation.de> <200801311433.38358.bernhard@intevation.de> <200801311459.13229.bernhard@intevation.de> Message-ID: <200801311606.45477.bernhard@intevation.de> On Thursday 31 January 2008 14:59, Bernhard Reiter wrote: > Now 2825, I've just realized that my change to rename the development dtd to a real one 1.2.1.dtd broke a number of tests. I will need to fix this before the release. Also in Debian sid the python-gdal package is currently broken, which also makes testing this harder for me. In short: There will be an rc2. ;) Bernhard -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080131/48ef4111/attachment.bin From bernhard at intevation.de Thu Jan 31 16:53:10 2008 From: bernhard at intevation.de (Bernhard Reiter) Date: Thu, 31 Jan 2008 16:53:10 +0100 Subject: 1.2.1rc1 = svn2825 In-Reply-To: <200801311606.45477.bernhard@intevation.de> References: <200801301129.01533.bernhard@intevation.de> <200801311459.13229.bernhard@intevation.de> <200801311606.45477.bernhard@intevation.de> Message-ID: <200801311653.11189.bernhard@intevation.de> On Thursday 31 January 2008 16:06, Bernhard Reiter wrote: > In short: There will be an rc2. ;) rev2826 has the dtd things fixed. Didrik meanwhile found a few problems with the mapserver plugin and plans to update the French translation tomorrow as well. Bernhard -- Managing Director - Owner: www.intevation.net (Free Software Company) Germany Coordinator: fsfeurope.org. Coordinator: www.Kolab-Konsortium.com. Intevation GmbH, Osnabr?ck, DE; Amtsgericht Osnabr?ck, HRB 18998 Gesch?ftsf?hrer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.intevation.de/pipermail/thuban-devel/attachments/20080131/432a3b03/attachment.bin