eXist journal - J.M. Vanel

howto test eXist  - eXist-1.0/test/report/html/index.html
http://drone.exist-db.org/log/existbot/existdb : log of eXist chat
http://www.zeesource.net/maps/map.do?group=335 : geographics map of eXist developers
http://wiki.exist-db.org/
IRC channel #existdb - [cvs] / exist / eXist-1.0 / src / org / exist
http://news.gmane.org/gmane.text.xml.exist - http://news.gmane.org/gmane.text.xml.exist.cvs

Contents


2006-03-20

Error in the request logger

[giulio] jean marc remember that there is a problem with xmlrpc and "big" documentsjmvanelYes , I 'll add that to the wiki page .No time to do more today and tomorrow .I propose this :
Potential problems
- there is a problem with XML-RPC when uploading "big" documents ( around 1Mb or even less ), e.g. using the GUI : the log seems to be incomplete
[giulio]no jean marc also with very little documents 10-15 kb
jmvanel
giulio , so the problem is when uploading any document with XML-RPC using one of the methods in
http://exist.sourceforge.net/devguide.html#N106BC ?
[giulio]jean marc the pronblem is not strictly to xmlrpc but how the message tcp is send when the document is not complete with a single tcp messagein general you can have problem with method exposebut how yesterday i write-->|wolf77 (n=opera@L28bf.l.pppool.de) has joined #existdb[giulio]the problem is that ceck the lenght of the message is not good

[giulio] is like you have a xml document with start node and no end node

jmvanel Giulio, when I look at the original RpcServlet , it just does
request.getInputStream()
and inputs that into xmlrpc.execute()

[giulio] yes and in this is correct

jmvanel Sothis is why I think that request.getInputStream() is able to gather physical message into a single logical message

[giulio] yes ok

the wrapper accept one message and pass it to xmlrpc server

but without know if this is complete or no

or also can be possible that the wrapper close the connection so nothing also arrive

jmvanel It may be that
is.read(contentBody);
in HttpServletRequestWrapper
doesn't real all and should be in a loop .

_wolf77_ chongma: you need to cast the object

jmvanel: yes, check for example RESTServer.getRequestContent

jmvanel OK.
Giulio , yes I think that's the problem. Could you try to replace
is.read(contentBody);
line 294 in HttpServletRequestWrapper
with
int bytes = 0;
int offset = 0;
int max = 4096;
while (( bytes = is.read( contentBody, offset, max )) != -1) {
offset += bytes;
}

2006-03-19

Extention of HTTP+requests+logger to other protocols

http://wiki.exist-db.org/space/HTTP+requests+logger

Test with WebDav

cadaver
dav:!> open http://localhost:8080/exist/webdav/db
Authentication required for exist on server `localhost':
Username: guest
Password:
dav:/exist/webdav/db/> ls
Listing collection `/exist/webdav/db/': succeeded.
Coll: XQTS 0 Mar 18 17:01
Coll: complex 0 Mar 17 20:27
Coll: test 0 Mar 17 20:28
Coll: testCollectionExists 0 Mar 18 19:00
Coll: �t� 0 Mar 18 18:10
dav:/exist/webdav/db/> put build.xmL
Uploading build.xmL to `/exist/webdav/db/build.xmL': Could not open file: No such file or directory
dav:/exist/webdav/db/> put build.xml
Uploading build.xml to `/exist/webdav/db/build.xml':
Progress: [=============================>] 100.0% of 56204 bytes succeeded.

2006-03-15

Tests with gcj library + eclipse

Only 5 errors left when linking  eXist to the gcj libraries !
Severity    Path    Resource    Description    Location    Creation Time    Id
Error    eXist-1.0/src/org/exist/client    DocumentView.java    The constructor OutputStreamWriter(FileOutputStream, Charset) is undefined    line 435    mars 15, 2006 7:23:20     3192
Error    eXist-1.0/src/org/exist/client    InteractiveClient.java    The constructor OutputStreamWriter(FileOutputStream, Charset) is undefined    line 2370    mars 15, 2006 7:23:19     3180
Error    eXist-1.0/src/org/exist/http/test    RESTServiceTest.java    BASE64Encoder cannot be resolved to a type    line 87    mars 15, 2006 7:22:38     2939
Error    eXist-1.0/src/org/exist/http/test    RESTServiceTest.java    The import sun cannot be resolved    line 44    mars 15, 2006 7:22:38     2938
Error    eXist-1.0/src/org/exist/xquery/value    TimeUtils.java    The method newDurationDayTime(boolean, BigInteger, BigInteger, BigInteger, BigDecimal) in the type DatatypeFactory is not applicable for the arguments (boolean, BigInteger, BigInteger, BigInteger, BigInteger)    line 95    mars 15, 2006 7:19:54     1639

So a test is possible with the gcj compiler too . Fedora Core 4 offers a convenient environment, the "native eclipse" .
But first I try with eXist compiled by normal eclipse, plus the GNU jre ( gcj libraries). There is a problem with Cocoon :
java.lang.reflect.InvocationTargetException
at java.lang.reflect.Method.invoke(java.lang.Object, java.lang.Object[]) (/usr/lib/libgcj.so.6.0.0)
at org.exist.start.Main.invokeMain(java.lang.ClassLoader, java.lang.String, java.lang.String[]) (Unknown Source)
at org.exist.start.Main.run(java.lang.String[]) (Unknown Source)
at org.exist.start.Main.main(java.lang.String[]) (Unknown Source)
at gnu.java.lang.MainThread.call_main() (/usr/lib/libgcj.so.6.0.0)
at gnu.java.lang.MainThread.run() (/usr/lib/libgcj.so.6.0.0)
Caused by: java.lang.NoClassDefFoundError: while resolving class: org.apache.cocoon.components.flow.javascript.fom.FOM_JavaScriptInterpreter
at java.lang.VMClassLoader.transformException(java.lang.Class, java.lang.Throwable) (/usr/lib/libgcj.so.6.0.0)
at java.lang.VMClassLoader.resolveClass(java.lang.Class) (/usr/lib/libgcj.so.6.0.0)
at java.lang.Class.initializeClass() (/usr/lib/libgcj.so.6.0.0)
at java.lang.Class.isAssignableFrom(java.lang.Class) (/usr/lib/libgcj.so.6.0.0)
at org.apache.avalon.excalibur.component.ComponentHandler.getComponentHandler(java.lang.String, java.lang.Class, org.apache.avalon.framework.configuration.Configuration, org.apache.avalon.framework.component.ComponentManager, org.apache.avalon.framework.context.Context, org.apache.avalon.excalibur.component.RoleManager, org.apache.avalon.excalibur.component.LogkitLoggerManager, org.apache.excalibur.instrument.InstrumentManager, java.lang.String) (Unknown Source)
This is caused by a class not found that I can't find nowhere ( in JRE or in eXist ) with :
for f in **/*.jar; do echo $f; unzip -l $f  | \
grep org.mozilla.javascript.tools.debugger.Main; done

So let's suppress Cocoon and test the eXist servet with :
http://localhost:8080/exist/xquery/guess.xql
To suppress Cocoon I comment out in web.xml :
    <servlet-mapping>
<servlet-name>Cocoon</servlet-name>
<url-pattern>/</url-pattern>
</servlet-mapping>
In fact I need to suppress the <servlet> tag too.
I had one more problem  with JAF, that I solved by adding this to the eclipse project libraries :
/usr/share/java/jaf.jar 

Youpeeeeeeeeeeeeeeeeee !
Exist ( i.e. guess.xql ) runs with the gcj libraies !
However, trying the sandbox
http://localhost:8080/exist/sandbox/sandbox.xql
with /* (and nothing in the DB) gives problems:
org.xml.sax.SAXParseException@53947f8
at org.exist.Indexer.fatalError(org.xml.sax.SAXParseException) (Unknown Source)
at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(java.lang.String, java.lang.String, org.apache.xerces.xni.parser.XMLParseException) (Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(org.apache.xerces.xni.XMLLocator, java.lang.String, java.lang.String, java.lang.Object[], short) (Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(java.lang.String, java.lang.String, java.lang.Object[], short) (Unknown Source)
at org.apache.xerces.impl.XMLScanner.reportFatalError(java.lang.String, java.lang.Object[]) (Unknown Source)
at org.apache.xerces.impl.XMLDocumentScannerImpl$PrologDispatcher.dispatch(boolean) (Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(boolean) (Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(boolean) (Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(org.apache.xerces.xni.parser.XMLInputSource) (Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(org.apache.xerces.xni.parser.XMLInputSource) (Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(org.xml.sax.InputSource) (Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(org.xml.sax.InputSource) (Unknown Source)
at org.exist.collections.Collection$4.run(org.exist.collections.IndexInfo) (Unknown Source)
at org.exist.collections.Collection.validateXMLResourceInternal(org.exist.storage.txn.Txn, org.exist.storage.DBBroker, java.lang.String, org.exist.collections.Collection$ValidateBlock) (Unknown Source)
at org.exist.collections.Collection.validateXMLResource(org.exist.storage.txn.Txn, org.exist.storage.DBBroker, java.lang.String, org.xml.sax.InputSource) (Unknown Source)
at org.exist.xmldb.LocalCollection.storeXMLResource(org.exist.xmldb.LocalXMLResource) (Unknown Source)
at org.exist.xmldb.LocalCollection.storeResource(org.xmldb.api.base.Resource, java.util.Date, java.util.Date) (Unknown Source)
at org.exist.xmldb.LocalCollection.storeResource(org.xmldb.api.base.Resource) (Unknown Source)
at org.exist.xquery.functions.xmldb.XMLDBLoadFromPattern.evalWithCollection(org.xmldb.api.base.Collection, org.exist.xquery.value.Sequence[], org.exist.xquery.value.Sequence) (Unknown Source)
at org.exist.xquery.functions.xmldb.XMLDBAbstractCollectionManipulator.eval(org.exist.xquery.value.Sequence[], org.exist.xquery.value.Sequence) (Unknown Source)
at org.exist.xquery.BasicFunction.eval(org.exist.xquery.value.Sequence, org.exist.xquery.value.Item) (Unknown Source)
at org.exist.xquery.InternalFunctionCall.eval(org.exist.xquery.value.Sequence, org.exist.xquery.value.Item) (Unknown Source)
at org.exist.xquery.PathExpr.eval(org.exist.xquery.value.Sequence, org.exist.xquery.value.Item) (Unknown Source)
at org.exist.xquery.DynamicCardinalityCheck.eval(org.exist.xquery.value.Sequence, org.exist.xquery.value.Item) (Unknown Source)
at org.exist.xquery.AbstractExpression.eval(org.exist.xquery.value.Sequence) (Unknown Source)
As you can see, the problem is also that the GNU jre ( gcj libraries) has no debugging informations .
What does eXist try to store ? Probably the sandbox/xml-highlight.xsl sylesheet .

This problem with the xerces parser seems to be well-know in the eXist team ?
The version I have in /usr/share/java is :
xerces-j2-2.6.2-4jpp_5fc
And eXist has Xerces 2.7.1

TO BE CONTINUED .

Extending Adam Retter (delirium)'s request logger

src/org/exist/http

Added Files:
Descriptor.java
Log Message:
Added the facility for a descriptor.xml file to detail path mappings for the REST interface.
Usefull for default document like functionality. Enables mappings of folder->file, folder->folder, file->folder, file->file

The replayer is in
tools/requestlog

2006-03-06

I'm debugging XQueryPool with XML-RPC , the GUI client, and always the same query .
I have the inmpression that the compiled query is never reused .
Also I don't understand the reason to have a Stack inside the values[] array .
 The test line 138 is never true :

 Back from a long discussion with my eXist client ...
[23:10:24] <wolf77> the last two questions are easy to answer:
[23:11:12] <wolf77> order by does not use an index. it would require a specific type of index that allows fast access to values by node ids.
[23:11:30] <wolf77> but I would be interested to see such an index added.

[23:12:12] <wolf77> the cluster was developed by a commercial company for a production system.
[23:12:20] <wolf77> I thus assume it is used in production.
[23:12:41] <[giulio]_x> yes wolf about the cluster in production
[23:12:56] <[giulio]_x> but I suspect the development is for a specific contenxt
[23:13:02] <[giulio]_x> like small documents
[23:13:44] <[giulio]_x> I write this because in my test i see that if the message send is not so big
[23:13:59] <[giulio]_x> the cluster work otherwise after a little there a re problem of syncro
[23:14:12] <wolf77> yes, that's very much possible. I also think they did not address queries to the database because they had a hardware proxy doing this.
[23:14:20] <[giulio]_x> yes
[23:14:45] <[giulio]_x> If I remember the developper write a day that have an hardware for do this

[23:17:48] <_jmvanel_> Thank you all.
[23:17:48] <_jmvanel_> And what about refining queries ?
[23:17:48] <_jmvanel_> First query :
[23:17:48] <_jmvanel_> /tag[constraint1]
[23:17:48] <_jmvanel_> Second query :
[23:17:48] <_jmvanel_> /tag[constraint1][constraint2]
[23:18:01] <_jmvanel_> is the first query kept in cache and reused ?
[23:18:17] <wolf77> Clear answer: no


Made some trials with AspectJ ( see my general blog ).

2006-02-22

Profiling with eclipse TPTP

Using the latest sub-release ( not daily build). One needs to download the eclipse plugin, plus the special OS-specific server . Then you have to go in "profiling" tab in window Run/profile , and do "Add" to add the analysis that you want . I chose:

I ran this query on the XQTS definition file :
declare namespace cat= ... ;
//cat:test-case [ @name="Literals002" ]
Here it the  time analysis result . The base time is the most interesting . Note that it took a very time, I was obliged to let it run all night.
Here are memory analysis:
Alas the export as CSV or HTML files crashed !

[09:21:37] <wolf77> I guess the journal and sync timings are explained by the query taking all night. eXist has probably done quite a lot of syncs over night ;-)
[09:21:53] <_jmvanel_> Yes, sure.
[09:21:57] <wolf77> Compare http://exist-db.org/files/profile.html
[09:21:58] <wolf77> Same query.
[09:22:41] <wolf77> As expected, most of the time is spent in NodeProxy.atomize(). No surprise: there's no index defined.
[09:22:51] <ljo> Well, just asking since you said it should not be used at all.
[09:23:14] <wolf77> Yeah, I was first wondering as well.
[09:36:26] <_jmvanel_> Comparing the results of YourKit , and mine ;-) , that is TPTP, I see that for
[09:36:26] <_jmvanel_> GeneralComparison.quickNodeSetCompare(Sequence)
[09:36:26] <_jmvanel_> The base time given TPTP is very low, whereas Yourkit shows 19% ; does this functions really just do
[09:36:26] <_jmvanel_> some delegation ?

2006-02-19

Versailles 2006 eXist meeting

My presentation ( OpenOffice.org 2 format ), My presentation in PDF
Dannes Wessels' (DiZzZz) presentation in PPT, DiZzZz) presentation in PDF
Wolfgang's two presentations : eXist architecture + history (in english) ; eXist algorithms (in german) .........
Pictures taken by DiZzZz http://dilemma.ow.nl/versailles2006/

My notes (saturday 18)

..........................

My notes (sunday 19)

PSVI - easier with JAXP
reference Schema cached - problem with non-existing schema makes eXist slow
distrib. without Cocoon ; doc. in the database;
new Ant targets: embedded eXist, plain server
SleepyCat better for data-centric apps
Wolf:
store 100000 files
potential for local optimisations in a lot of places
commit only bugs on the HEAD branch

test

creating small data by program, and storing it 100000 times
Perig:
use XQTS data structure, submit our tests to XQTS
need low-level tests
Wolf:
transaction tests also tests BFile
transform tests into stress tests by adding parametrizable loops
tests regressions in XQuery: store in the DB the reference profiles, and compare the to the actual profile
JUnit 4 : overall setUp and tearDown
TestNJ

using Java 5 features and generating Java 1.4 byte code ?

making conf.xml dynamic
Spring use: configuration ; class instantiation
pool feature?

NodeNumbering interferface for DLN
key: collection id, real key --> node id

basic design decision in eXist : node set versus traversal
Jaxen?
eXist: context + input node set --> result
reused from Saxon: serialization, functions implementation

Prague: 5 hours 9-2 o'clock
Wiki page;
development team : one hour, roadmap

orbeon 3 + javascript
Xforms implementation in XQuery : not full, needs documentation
XQuery inside Xforms
MVC pattern : controller = XQuery
Perig: explore XUL runner (Mozilla)

september : Monet conference, in-memory DB, meta-language internally, XML : 2 id's internally

documentation : extended XQ features, modules TODO,
Webdav DONE
TODO conf.xml ; enhance indexing, using XQdoc

sf compile farm ; continous build system

XQTS is not quite finished, but it was a team effort, with Wolfgang and me writing the main script,
 and Perig and Dannes writing a new function to define a context by program.



2006-02-17

Preparing eXist meeting ...

java -cp tools/lib/jdepend-2.9.jar jdepend/swingui/JDepend build/classes


Added this at begining of /etc/ant.conf
# jmv
return

2006-02-16

do you know what schemes wolf is looking at?
[10:00:55] <_jmvanel_> http://dbs.uni-leipzig.de/en/projekte/XML/Boehme_DLN_DIWeb_CR.pdf
DLN = dynamic level numbering : Hourrah, Wolfgang commited it !

howto work with the "new numbering" CVS branch

I went to "configure tags" when in CVS repository exploring,
I navigated to NativeBroker, and I added "DLN" to the list of remembered tags.

Then I created a new eclipse project from the usual eXist-1.0 first level directory, but with the branch tag DLN .
 
Note that this CVS branch (and hence the project) contains only the directory src/ , no lib/ , no webapp/ , etc .

So I made the new eclipse project ( I called it eXist-1.0_dln) dependant on the normal eclipse project "eXist-1.0" . But that 's not enough to get the libs from the normal eclipse project "eXist-1.0" . You must also add all third party libs from the normal eclipse project to the exported libraries in tab "Java build path / order and export" .
Then all should compile .

Note that this way Wolfgang is not obliged to put all sources in the branch DLN , as he done now . He could put on the new branch just the modified files .

Then to execute with the GUI client in embedded mode , I must copy or link the missing files :
# linking or creating necessary files from the CVS HEAD:
mkdir --parents webapp/WEB-INF/data/
ln -s $exist/conf.xml .
ln -s $exist/client.properties .
ln -s $exist/webapp/WEB-INF/catalog.xml .
ln -s $exist/webapp/WEB-INF/log4j.xml .
Variable $exist represents the location of the normal eclipse project .

That's all .



Using this to kompare the old and new code :
kompare src ../eXist-1.0-sync/src &



My fisrt stack with the new numbering scheme :
Reading a comment when storing hamlet.xml :
java.lang.NullPointerException
    at org.exist.dom.CommentImpl.serialize(CommentImpl.java:55)
    at org.exist.storage.NativeBroker$15.start(NativeBroker.java:2247)
    at org.exist.storage.dom.DOMTransaction.run(DOMTransaction.java:56)
    at org.exist.storage.NativeBroker.storeNode(NativeBroker.java:2263)
    at org.exist.storage.DBBroker.storeNode(DBBroker.java:483)
    at org.exist.Indexer.comment(Indexer.java:219)

Allready solved by added 1 line in Indexer !


[16:52:35] <wolf77> Jean-Marc: queries don't work with the new numbering scheme yet.
[16:52:52] <wolf77> all the NodeSet methods need to be changed first.
[16:53:05] <wolf77> all you can do now is to store/retrieve documents.
[17:00:21] <_delirium> wolf77: hey wolf, did you have a chance to look at the HTTP 500 error problems in that email I sent you?
[17:00:38] <wolf77> also, parts of the indexes do already store the new node ids, other parts still use the old one. you can't expect that to work ;-)
[17:02:25] <wolf77> _jmvanel_: I calculate at least another 8 weeks until the test suite will run with the new scheme.

Adding to the "Incompatible changes", in the change log.

[19:15:27] <_jmvanel_> Want to add "no more implicitly global variables"
[19:15:27] <_jmvanel_> to "Incompatible changes", in the change log.
[19:15:27] <_jmvanel_> Was that on the 24th snapshot too ?


before eXist would let you write $a[1 to 10], you now need to write $a[(1 to 10)]

2006-02-13

Commited feature "XQuery trigger with update" (not quite finished)





09:30:29 _jmvanel_ Wolf, debugging in Modification.selectAndLock() , lines:
09:30:30 _jmvanel_ NodeList nl = select(docs);
09:30:30 _jmvanel_ lockedDocuments = ((NodeSet) nl).getDocumentSet();
09:30:30 _jmvanel_ nl has just one node, but
09:30:30 _jmvanel_ getDocumentSet() returns its cachedDocuments , that includes all collection.
09:30:30 _jmvanel_ I think it's not normal.
09:32:38 wolf77 Yes, that's rather strange.
09:36:32 _jmvanel_ Can be seen in XQueryTriggerTest.testUpdateDocument() , but with the new
09:36:33 _jmvanel_ update Trigger on my site .

My breakpoints :
Modification [line: 101] - nullDocumentTrigger
Modification [line: 214] - selectAndLock()
Modification [line: 283] - getTrigger(DocumentImpl)
TriggerStatePerThread [line: 125] - setTriggerRunningState(int, DocumentTrigger, DocumentImpl)


2006-02-05


[20:39:27] <_jmvanel_> About to add this field in XQueryTrigger
[20:39:27] <_jmvanel_> private static ThreadLocal triggerRunning = new ThreadLocal() {
         protected synchronized Object initialValue() {
return Boolean.FALSE;
}
[20:39:27] <_jmvanel_>
[20:39:27] <_jmvanel_> and this call at the beginning of XQueryTrigger.prepare()
[20:39:27] <_jmvanel_> triggerRunning.set(Boolean.TRUE);
[20:39:27] <_jmvanel_> and the converse at the end of  finish()

2006-01-28


appendChildren(Txn transaction, NodeList nodes, int child)

[[13:39:53] <Perig> Well... trying to adress this XUpdate bug
[13:40:21] <Perig> Interesting one whic will probably lead us in the very core of the code :-)
[13:41:13] <_jmvanel_> I was also looking at http://skaringa.sourceforge.net/doc.html#i21219507
[13:41:37] <[giulio]_> perig the "yes" about omit xquery diagnostics is for me?
[13:41:45] <_jmvanel_> How could I help Perig ?
[13:42:04] <Perig> giulio : what are you talking about ?
[13:42:17] <Perig> _jmvanel_ in writing a simple test case :-)
[14:47:45] <_jmvanel_> Where in the W3C docs do they say that adjacent nodes are necessarily glued as one ?
[14:49:47] <[giulio]_> sorry Perig I'am in wrong
[14:50:22] <Perig> Looking for the spec to make everything clear
[14:52:47] <_jmvanel_> I remember that the spec is tolerant about that ; the glueing may occur or not .
[14:53:06] <_jmvanel_> But not 100% shure.
[14:53:07] <Perig> http://www.w3.org/TR/xpath#section-Text-Nodes
[14:53:17] <Perig> Old Xpath specs.
[14:53:43] <Perig> Trying to get the up to date data model
[14:54:28] <Perig> OK. Clear :
[14:54:30] <Perig> http://www.w3.org/TR/xpath-datamodel/#TextNode
[14:55:20] <Perig> So, the user (and myself) is right IMHO. Do you agree ?
...
[15:02:05] <_jmvanel_> Perig of course you're right about the meaning of the spec regarding adjacent text nodes: they are forbiden.
[15:02:11] <Perig> OK. But I'm trying to find a definitive solution
15:03:14] <_jmvanel_> I can write a JUnit test method; do you have a preference for an existing test class ?
[15:03:25] <Perig> In clear, eXist sould *never ever* allow sibling text nodes.
[15:03:58] <_jmvanel_> Might be difficult but yes that's the goal.
[15:04:33] <Perig> Major deficiency IMHO. Node IDs are precious :-)
[15:05:18] <Perig> ElementImpl seems to be a good candidate though
[15:07:07] <Perig> (trigger a ReplaceChild rather than an AppendChild)
[15:07:19] <_jmvanel_> naively I would say that glueing nodes is easier than splitting, because we don't consumme precious ID intervals.
[15:07:36] <_jmvanel_> Which line ?
[15:08:02] <Perig> ElementImpl, line 449
[15:08:34] <Perig> but 438 and 446 may be concerned as well.
[15:09:30] <Perig> coalescing nodes it not that easy (even if there are more difficult things) because we must test that we have text nodes...
[15:09:52] <Perig> whereas eXist tries to avoid such tests
[15:11:29] <_jmvanel_> I see. Have you looked at TextImpl ?
[15:12:00] <Perig> Not yet. Still wondering who should have control :-)
[15:12:23] <_jmvanel_> BTW, don't you think we should once and for all apply eclipse format on whole eXist and commit that ?
[15:12:39] <_jmvanel_> Democratically , of course ...
[15:12:47] <Perig> In short : should the parent check that it doesn not have 2 consecutive children ?
[15:13:06] <Perig> Or... the node that it does not have a sibling ?
[15:13:13] <_jmvanel_> These lines with 119 characters are bothering .
[15:13:43] <Perig> Jean Marc : I'd support such an initiative.
[15:13:51] <_jmvanel_> The latter avoids the type test ...
[15:13:59] <Perig> But I'am rather used to "Apache" coding rules.
[15:15:36] <_jmvanel_> OK. interesting , but let's settle that after ...
[15:16:14] <Perig> No problem : I want to take some time with this text node problem.
[15:16:39] <_jmvanel_> I'll have a look at TextImpl, but I must tell that we have a similat class hierarchy for memtree, alas.
[15:17:09] <Perig> I'm also used to these : http://www.gnu.org/software/classpath/docs/hacking.html#SEC6
[15:17:41] <Perig> Sure : we clearly have 2 parallel hierarchies
[15:18:10] <Perig> Nice place for design pattern governed refactorings ;-)
[15:18:41] <_jmvanel_> I remember that Wolf wrote that he got inspiration from Saxon.
[15:19:10] <_jmvanel_> But why not reuse Saxon for the in-memory trees?
[15:19:25] <Perig> he, he ;-)
[15:19:46] <_jmvanel_> Some discussions lately that I missed ?
[15:20:38] <Perig> No strategic ones.
[15:21:03] <Perig> Just that wolf has a milestone about developing a more robust numbering scheme
[15:21:28] <_jmvanel_> Anyway, bebore being intelligent, let me write my JUnit ; so in case of need, I'll be able to debug ...
[15:22:40] <_jmvanel_> About the  numbering scheme , I saw and article about the ReiserFS File System in Linux Magazine (en Français).
[15:22:53] <_jmvanel_> Maybe some ideas to pick ...
[15:26:35] <Perig> Wolfgang told me that he was relyng on variable lenghth keys.
[15:27:09] <Perig> What I'd like is (yet another) pluggable thing.
[15:30:05] <_jmvanel_> :-)
[15:30:10] _delirium [n=adam@cpc1-derb1-6-0-cust26.leic.cable.ntl.com] a quitté IRC : "using sirc version 2.211+KSIRC/1.3.12"
[15:30:51] <Perig> Gev made a suggestion that at first glance looked crazy : store all the possible relationships between nodes.
[15:31:20] <Perig> However... storing them in a cache beginds to make sense
[15:32:55] <Perig> A plugabble thing would also ease what has been requested recently on the mailing-list : traverse XIncludes.
[15:34:16] <Perig> That is, in a given execution scope, node X from document A is to be viewed as the parent of node Y in docment B
[15:35:04] <_jmvanel_> :-) flexibility points sure guide the design .
[15:35:26] <Perig> I hope they will...
[15:36:50] <_jmvanel_> Another idea : uncouple the storage and the indexing, so that one can keep the XML, e.g., in a plain file system, with indices that point to interesting places in this storage.
[15:37:23] <Perig> Oh yes :-)
[15:38:03] <Perig> A design close to SDX's BTW
[15:38:15] <_jmvanel_> haha
[15:38:44] <_jmvanel_> So we need abstractions of :
[15:38:51] <_jmvanel_> - storage
[15:38:56] <_jmvanel_> - indexer
[15:39:29] <_jmvanel_> - editable storage
[15:40:14] <Perig> indexerSSSS
[15:40:54] <Perig> an easy abstraction would be to consider stored item as a ache entry
[15:41:05] <Perig> That how Gev's application works :-)
[15:46:11] <Perig> BTW : anybody remembers that "binary" XML storage (from China ?) that store the hierarchy in the file itself ?
[15:48:01] <_jmvanel_> No.  What is  " a ache entry" ?
[15:49:15] <Perig> cache
[15:50:21] <Perig> In clear : use EH cache in place of the native BFiles :-)
[15:52:47] <[giulio]_> so if the file is not changed there are no serilization?
[15:53:29] <_jmvanel_> I know nothing yet about EH cache, except that it's widely used.
[15:54:09] <Perig> Yes : now used by Cocoon
[15:55:16] <_jmvanel_> The least we can do is having a clean separation between "low-level" storage and XML storage, which brings as back to the indexing scheme ...
[15:55:34] <_jmvanel_> us back
[15:56:08] <Perig> Yes. The StoredNode/NodeProxy decoupling is a good step forward
[15:58:07] <_jmvanel_> yes.
[16:01:36] <Perig> But NodeProxy has still an internalAddress :-(
[16:03:01] <_jmvanel_> Alas.
[16:03:01] <_jmvanel_>
[16:03:01] <_jmvanel_> Maybe some ideas can also be gained from Jaxen :
[16:03:01] <_jmvanel_> http://jaxen.codehaus.org/faq.html
[16:03:01] <_jmvanel_> I just read the book about PMD; they use Jaxen to query by XPath the Abstract Syntax Tree.
[16:03:01] <_jmvanel_> The AST does not implement any DOM or Jax interfaces, but iterators for
[16:03:01] <_jmvanel_> the XPath axis .
[16:24:28] <_jmvanel_> I propose to put this in XPathQueryTest Javadoc:
[16:24:28] <_jmvanel_> tests involving only XPath expressions,
[16:24:28] <_jmvanel_> and in XQueryTest
[16:24:28] <_jmvanel_> tests involving the others constructs of the XQuery language
[16:24:28] <_jmvanel_> besides XPath expressions
[16:24:28] <_jmvanel_> public class XQueryTest
[16:25:56] <Perig> Some serious clean ups there ;-)
[16:28:36] <_jmvanel_> But do you agree with the classification principle ?
[16:30:58] <Perig> Yes (although I think that all this kind of tests should be handled by the XQTS)
[16:32:58] <_jmvanel_> XQTS ?
[16:45:22] <_jmvanel_> Anyway, Perig, we got our bugtest in XQueryTest.java ...
[16:51:50] Perig [n=chatzill@vil35-2-82-236-174-106.fbx.proxad.net] a quitté IRC : Read error: 113 (No route to host)
[16:53:48] <_jmvanel_> When we fix thzt, we can correct the misspelling:
[16:53:48] <_jmvanel_> bugtestXUpdateWithAdvancentTextNodes() ;-)
[16:54:16] <_jmvanel_> Adjacent
[16:57:27] <_jmvanel_> No least than 6 functions to add one or more child in ElementImpl :-( !
[17:16:42] <_jmvanel_> Still trying to orientate in the ElementImpl.appendXXX() jungle;
[17:16:42] <_jmvanel_> as a boussole I have the call graph.
[17:16:42] <_jmvanel_> But my laptop screen is too small, have use my 1600x1200 screen !
[17:17:36] <_jmvanel_> And ElementImp is 1500 lines, should be split as least in 3 !
[17:31:42] <_jmvanel_> To see all the little white stones you've put ... Nice!
[17:31:42] <_jmvanel_> When will there be an oportunity to review all that with Wolf in flesh ?
[17:32:05] <Perig> :-)
[17:33:07] <_jmvanel_> Will you be in Prague ?


2006-01-25

I saw nice refactorings in the storage/ package :
I tried to make a JUnit test for BTree. It 's not obvious, because of configuration that must be in a file, and of the undocumented methods, e.g. addValue() .

2005-12-07

Problems with French accents in HTTP GET

When I do this request :
http://localhost:8080/exist/xquery/test-param.xql?param=%E9t%E9

with this simple XQuery test-param.xql :
(: declare option exist:encoding "UTF-8"; "iso-8859-1"; :)
declare variable $param { request:request-parameter("param", "") };
<html>{ $param }</html>

I get this result :
<html>ᅵtᅵ</html>

When debugging, I saw in HttpRequestWrapper.decode()
that  containerEncoding == formEncoding == UTF8

Here is how my string 'été' is when it goes out of the J2EE function HttpServletRequest.getParameterValues(key)
[17:00:14] <|jmvanel|>      (int)values[0].charAt(0)
[17:00:14] <|jmvanel|>      (int) 65533
[17:00:14] <|jmvanel|>      (int)values[0].charAt(1)
[17:00:14] <|jmvanel|>      (int) 116
[17:00:14] <|jmvanel|>      (int)values[0].charAt(2)
[17:00:14] <|jmvanel|>      (int) 65533
Don't know if it's normal, but looks like UTF16

I came to this because my main problem comes in a "real" XQuery when I compare some content in the DB with the string from the HTTP parameter .

When I try this with param=é everything is normal ; result is "true" :
declare variable $param { request:request-parameter("param", "") };
<html>{
<param>{$param}</param> ,
"e accent ", "'é'" ,
"e accent eq $param",
"é" eq $param
}</html>
The page returned is UTF-8 , text/html . One can see again that the 2 é are not output right .

2005-11-25

I have corrected some bugs on my Web test GUI .
It's really useful for me .
My web test gui is a CVS module inside eXist , so, to download it :
cvs -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/exist login
cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/exist co -P web_test_gui
    There's no build.xml . I use eclipse; the eclipse project is in CVS .
    The main class is WebTestFrame. The CVSWeb is here :
http://cvs.sourceforge.net/viewcvs.py/exist/web_test_gui/src/org/exist/webtest/

The next thing I want to add are some user comments on each test URL , and while doing that I will create a class TestURL, now it's just a String .

2005-11-12

Non-regression GUI tester for multiple URLs

I commited my my non-regression GUI tester; see more details below .

To use it ;

cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/exist co web_test_gui
Optionaly create a new eclipse project from the directory web_test_gui
Start the class org/exist/webtest/WebTestFrame

if one leaves the "source directory" empty, no management of original source file will be done .

2005-11-08

Interesting thread on the list :
Re: [Exist-open] eXist vs SleepycatMichael Beddow replies longly on this .

Non-regression GUI tester for multiple URLs

I did a non-regression tester for multiple URLs implemented by XQuery source.
I works for me ; If some of you find it useful, I can share it as a new module in eXist CVS, or elsewhere. I used it to :
Screenshot :
webtest-gui.png
Download eclipse auto-sufficient project :
webtest.zip  - uptodate version is in eXist CVS :

cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/exist co web_test_gui

What exactly can I do with it?
    You can enter some relative URLs ; then the program will save the original XQuery source and result. After that you can modify (optimize or refactor) your xqueries , and push "check all URLs", and XMLUnit checks all results .
Jwhat do I have to add into "Relative test URL"?


Look the tooltip :-)


can be terninated or not by / ,


and the relative URL's must begin or not by / accordingly .



look please httt://www.rosavtotorg.ru

wolf77 yeah, it's getting nicer and nicer ;-) The map selection is cool.

gev thanks

jmvanel Yes, gev, it looks good ; is it a 100% eXist application, no JSP, no JDBC, no LDAP ?

giulio692 Gev very nice and fast

gev yes it's 100% eXist application. I only add batik library and cocoon's svg generator to rasterize svg images

gev also I use cocoon's features to scale images

2005-10-20

count ( tokenize( "", "") )
returns 1 !!!!!!!!!!!

            <!-- jmv -->
<map:match pattern="*/**">
<map:mount check-reload="yes" src="{1}/sitemap.xmap"
uri-prefix="{1}" />
</map:match>


When starting GUI client in local mode :

Caused by: org.exist.EXistException: org.exist.storage.btree.DBException: Failed to open database file: /mnt/usr/src/usr2/eXist-1.0/webapp/WEB-INF/data/dom.dbx. It is locked by another process.
        at org.exist.storage.NativeBroker.<init>(NativeBroker.java:249)
        at org.exist.storage.BrokerFactory.getInstance(BrokerFactory.java:34)
        at org.exist.storage.BrokerPool.createBroker(BrokerPool.java:855)
        at org.exist.storage.BrokerPool.initialize(BrokerPool.java:606)
        at org.exist.storage.BrokerPool.<init>(BrokerPool.java:542)
        at org.exist.storage.BrokerPool.configure(BrokerPool.java:158)
        at org.exist.xmldb.DatabaseImpl.configure(DatabaseImpl.java:128)


DOMFile(Paged).setFile(File) line: 392
DOMFile.<init>(BrokerPool, File, CacheManager) line: 180
NativeBroker.<init>(BrokerPool, Configuration) line: 215
BrokerFactory.getInstance(BrokerPool, Configuration) line: 34

            FileChannel channel = raf.getChannel();           
            if (channel.tryLock() == null)
                throw new DBException("Failed to open database file: " + file.getAbsolutePath() +
                        ". It is locked by another process.");

When starting GUI client :

org.xmldb.api.base.XMLDBException: org.apache.xmlrpc.XmlRpcException: Invalid character data corresponding to XML entity &#16;
    at org.exist.xmldb.RemoteCollection.getResource(RemoteCollection.java:290)
    at org.exist.client.InteractiveClient.getResources(InteractiveClient.java:320)
    at org.exist.client.InteractiveClient.run(InteractiveClient.java:2139)
    at org.exist.client.InteractiveClient.main(InteractiveClient.java:222)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at org.exist.start.Main.invokeMain(Main.java:119)
    at org.exist.start.Main.run(Main.java:372)
    at org.exist.start.Main.main(Main.java:48)
Caused by: org.apache.xmlrpc.XmlRpcException: org.apache.xmlrpc.XmlRpcException: Invalid character data corresponding to XML entity &#16;
    at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java:457)
    at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:163)
    at org.exist.xmldb.RemoteCollection.getResource(RemoteCollection.java:288)

2005-10-17

This week -end I switched from Mandriva to Fedora Core 4 .

08:32 
What's interesting is that there is the latest gcj Java development kit with native eclipse . So I created a project from the eXist sources .
And guess what ?
It runs without problem? ;)
I will probably !
But just 6 compilation problems !

Severity    Description    Resource    In Folder    Location    Creation Time
2    The constructor OutputStreamWriter(FileOutputStream, Charset) is undefined    DocumentView.java    eXist1.0/src/org/exist/client    line 207    October 17, 2005 7:07:46 AM
2    The constructor OutputStreamWriter(FileOutputStream, Charset) is undefined    InteractiveClient.java    eXist1.0/src/org/exist/client    line 2428    October 17, 2005 7:07:46 AM
2    The import sun cannot be resolved    ForExpr.java    eXist1.0/src/org/exist/xquery    line 39    October 17, 2005 7:07:24 AM
2    The method newDurationDayTime(boolean, BigInteger, BigInteger, BigInteger, BigDecimal) in the type DatatypeFactory is not applicable for the arguments (boolean, BigInteger, BigInteger, BigInteger, BigInteger)    TimeUtils.java    eXist1.0/src/org/exist/xquery/value    line 95    October 17, 2005 7:06:36 AM
2    Override cannot be resolved to a type    UntypedAtomicValue.java    eXist1.0/src/org/exist/xquery/value    line 47    October 17, 2005 7:06:36 AM
2    Syntax error, annotations are only available if source level is 5.0    UntypedAtomicValue.java    eXist1.0/src/org/exist/xquery/value    line 47    October 17, 2005 7:06:36 AM

2005-09-26

Some job for eXist ?
template based search in eclipse
https://bugs.eclipse.org/bugs/show_bug.cgi?id=110157

2005-09-25


Refactoring : adding NodeProxy.TO_BE_COMPUTED in calls to parentWithChild()

Good breakpoint for all cases :
PathExpr(AbstractExpression).eval(Sequence) line: 43


Added support for HTTP URL's in fn:doc() . Works partly .....

Caused by: java.lang.ClassCastException: org.exist.memtree.DocumentImpl
    at org.exist.xquery.value.ValueSequence.toNodeSet(ValueSequence.java:163)
    at org.exist.memtree.NodeImpl.toNodeSet(NodeImpl.java:523)
    at org.exist.xquery.LocationStep.eval(LocationStep.java:154)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:43)
    at org.exist.xquery.PathExpr.eval(PathExpr.java:159)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:43)
    at org.exist.xquery.XQuery.execute(XQuery.java:136)
    at org.exist.xmldb.LocalXPathQueryService.execute(LocalXPathQueryService.java:330)


2005-09-24

I chase the -1 ;-)

DOCUMENT_NODE_ID = -1


In class NodeProxy: no javadoc on this :
What's the meaning of argument level == -1 ?
    /* (non-Javadoc)
     * @see org.exist.dom.NodeSet#parentWithChild(org.exist.dom.DocumentImpl, long, boolean, boolean, int)
     */
    public NodeProxy parentWithChild(
            DocumentImpl otherDoc,
            long otherId,
            boolean directParent,
            boolean includeSelf,
            int level)


NodeProxy

Thread [Thread-7] (Suspended)
NodeProxy.getInternalAddress() line: 313
DOMFile.getNodeValue(NodeProxy, boolean) line: 1735
NativeBroker$7.start() line: 1619
NativeBroker$7(DOMTransaction).run() line: 56
NativeBroker.getNodeValue(NodeProxy, boolean) line: 1622
NodeProxy.getNodeValue() line: 289
NodeProxy.convertTo(int) line: 521
FunString.eval(Sequence, Item) line: 71
PathExpr.eval(Sequence, Item) line: 144
PathExpr(AbstractExpression).eval(Sequence) line: 43
XQuery.execute(CompiledXQuery, Sequence) line: 136
LocalXPathQueryService.execute(String[], NodeSet, CompiledExpression, String) line: 330
LocalXPathQueryService.execute(CompiledExpression) line: 131
QueryDialog$QueryThread.run() line: 397

The stack :
Thread [Thread-9] (Suspended)
DOMFile.findValue(Object,NodeProxy) line: 1087
DOMFile.getNodeValue(NodeProxy,boolean) line: 1737
NativeBroker$7.start()line: 1619
NativeBroker$7(DOMTransaction).run()line: 56
NativeBroker.getNodeValue(NodeProxy,boolean) line: 1622
NodeProxy.getNodeValue()line: 289
NodeProxy.convertTo(int)line: 521
FunString.eval(Sequence,Item) line: 71
PathExpr.eval(Sequence,Item) line: 144
PathExpr(AbstractExpression).eval(Sequence)line: 43
XQuery.execute(CompiledXQuery,Sequence) line: 136
LocalXPathQueryService.execute(String[],NodeSet, CompiledExpression, String) line: 330
LocalXPathQueryService.execute(CompiledExpression)line: 131
QueryDialog$QueryThread.run()line: 397

2005-09-23


Misleading error message:
declare variable $workflows-conf as xs:string
{ document( concat($projectURI, '/workflow.xconf' ) ) };
entails:
Type error: the sequence cannot be converted into a node set. 
Item type is xs:string
The exact contrary ! TODO check with latest CVS .

Another:
declare variable $workflows-conf as element()
    { document( concat($projectURI, '/workflow.xconf' ) ) };
entails:
The type of variable $workflows-conf does not match the declared type: element. Got: element.

2005-09-22


0. start server.sh
1. in the GUI create a new collection /db/projects
2. fill it with tens of files
3. look at http://localhost:8080/exist/admin/admin.xql?panel=browse&collection=/db/projects
    exception while retrieving collection: Collection /db/projects not found [at line 17, column 24]
4. restart server.sh
5. refresh   http://localhost:8080/exist/admin/admin.xql?panel=browse&collection=/db/projects
    OK !?!

Not reproduced by Wolfgang and Giulio .



org.xmldb.api.base.XMLDBException
    at org.exist.xmldb.LocalXMLResource.getContent(LocalXMLResource.java:142)
    at org.exist.client.ClientFrame$TableMouseListener.mouseClicked(ClientFrame.java:1261)
    at java.awt.AWTEventMulticaster.mouseClicked(AWTEventMulticaster.java:212)


    jmvanel    At line 120 in process.xq I propose instead of :

let $contains :=
for $ss in $list
return
if ( $ss = $s ) then true
else ""

===>
let $contains := some $ss in $list satisfies $ss eq $s

How do I get an XQuery call Stack ?
I see something in XPathException ...
    -->|    erichanson (n=erichans@c-24-22-56-63.hsd1.or.comcast.net) has joined #existdb
    <--|    erichanson has left #existdb
    jmvanel    OK. I saw that it's done in class FunctionCall .
I suppose that , if I don't get a call Stack, my error is not a function call, but at the root level ...
    jmvanel    Going out for a walk ...
    wolf77    you can see a call stack if the XPathException was created with an XQueryAST node.
    wolf77    i.e. new XPathException(getASTNode(), ...)
    wolf77    but some objects, like values, don't have an AST node.
    wolf77    In these cases, the top-level expression can catch the exception and add its own AST. This is what FunctionCall does.
    jmvanel    OK. The old code has not new XPathException(getASTNode(), ...)
but it would be useful to add it when we come across it . Correct ?
    wolf77    yes, exactly.
    jmvanel    And what about activating the Profiler class ?
    wolf77    You can activate it with an option in the xquery:
    wolf77    declare option exist:profiling "enabled=yes verbosity=5";

Where could I look to solve this bug ?

let $v as element()* := ( <assign/> , <assign/> )
let $w := <r>{ $v }</r>
let $x as element()* := $w/assign
return $x

Error found: The type of variable $x does not match the declared type: element. Got: element..
    jmvanel_    suppress the 2nd as element() and no error ...
    jmvanel_    Also happens when $w is a node from the DB .


2005-09-21

I'm doing Micheal Kay's tutorial with eXist:
http://www.stylusstudio.com/xquery_primer.html

doc('http://www.stylusstudio.com/examples/videos.xml')
Found 0 items. Compilation: 80ms, Execution: 7ms :-(


Conversation on #drools

    conan    exist is just two nots
    jmvanel    Sorry , two nots = ?
    -->|    mic_neale (~mic_nea@ppp101-181.lns1.bne1.internode.on.net) has joined #drools
    conan    yes
    conan    in Rete
    conan    an exist is actually two Nots
    conan    Not checks the the non existence of a fact
    conan    so two nots, one after the other, check for existence.
    jmvanel    The idea is to use drools as a rewriter/optimiser for a programming language.
The source code is avaiable as an AST (Abstract Syntaxic tree) of Java objects .
    conan    ok
    conan    drools 3.0 would probably be very usefull for that
    conan    especially the module and truth maintenance
    conan    truth maintenace means that one fact depends upon another.
    conan    so if a rule fires and that consequence asserts a fact, that fact can depend on the rule that fired it. if you remove the facts that fired the original rule it also removes the asserted facts.
    conan    modules give a kind of controlled "flow".
    conan    each rule is associated with a Module. the agenda, which controlls the firing, has a stack of "modules". you set the current module which places it ontop of that stack. only rules in that current module will fire.
    conan    so you can have a load of rules in a "validation" module and another load of rles in a "process" module. Even though all rules might be true at the same time you can make sure all validation ones are evaluated first.
    conan    you will also be able to check for the complete non existence of a fact/condition in the WorkingMemory, or vice versa if a fact/condition exists.

    jmvanel    But without the thruth maintenance (validation is allready done), can't I use current Drools.
I have simple rules like this :
Source code:
/ root [ key = 123 ]
AND index present on key
==> rewrite as :
$key := util:qname-index-lookup( xs:QName("key"), 123 )
$root := $key / parent::root

    conan    jmvanel: sure if your rules are simple enough you can use 2.0


2005-09-20

bug on satisfies

I investigate the bug on satisfies .
* Quantified expressions behave strangely
(Bug #1267554 - http://sourceforge.net/tracker/index.php?func=detail&aid=1267554&group_id=17691&atid=117691)
The evaluation of quantified expressions ('[some|every] $foo in [expr] satisfies [expr]') seems to behave incorrectly.  For example the code fragments

--------
every $foo in (1,2,3) satisfies
    let $bar := baz
    return false()
--------

and

--------
declare function local:foo() { false() };

every $bar in (1,2,3) satisfies
    local:foo()
--------

both return true, whereas I would expect them to return false.

The original code in class QuantifiedExpression :
        for(SequenceIterator i = inSeq.iterate(); i.hasNext(); ) {
contextItem = i.nextItem();
var.setValue(contextItem.toSequence());
var.checkType();
satisfiesSeq = returnExpr.eval(contextSequence);
// LOG.debug("satisfies returned: " + satisfiesSeq.getLength());
if(returnExpr.returnsType() == Type.BOOLEAN)
found = satisfiesSeq.effectiveBooleanValue();
else
found = satisfiesSeq.getLength() != 0;
if((mode == SOME && found) || (mode == EVERY && !found))
break;
}
context.popLocalVariables(mark);
return found ? BooleanValue.TRUE : BooleanValue.FALSE;
Proposed modification by Sam Stokes :
        for(SequenceIterator i = inSeq.iterate(); i.hasNext(); ) {
contextItem = i.nextItem();
var.setValue(contextItem.toSequence());
var.checkType();
satisfiesSeq = returnExpr.eval(contextSequence);
// LOG.debug("satisfies returned: " + satisfiesSeq.getLength());

found = satisfiesSeq.effectiveBooleanValue();
if((mode == SOME && found) || (mode == EVERY && !found))
break;
}
context.popLocalVariables(mark);
return found ? BooleanValue.TRUE : BooleanValue.FALSE;


There is no test for the feature :
grep satisfies  src/**/test/*.java
return nothing .

Debugging

every $foo in (1,2,3) satisfies
    let $bar := baz
        return    false()

At first loop iteration :
returnExpr.returnsType()
     (int) 11
i.e. Type.ITEM
     
satisfiesSeq.getLength()
     (int) 1

satisfiesSeq.effectiveBooleanValue()
 (boolean) false

satisfiesSeq is a ValueSequence whose value has type BooleanValue .

Code reading

Proposed code is simpler. It relies on function effectiveBooleanValue() that has the desired behavior . Probably earlier the  function effectiveBooleanValue() t was not reliable, so the extra test in current version. I think we can accept the patch.

bug on conversion to boolean

Thread [main] (Suspended (breakpoint at line 185 in DoubleValue))
DoubleValue.effectiveBooleanValue() line: 185
FunBoolean.eval(Sequence, Item) line: 62
EnclosedExpr(PathExpr).eval(Sequence, Item) line: 144
EnclosedExpr.eval(Sequence, Item) line: 58
PathExpr.eval(Sequence, Item) line: 144
ElementConstructor.eval(Sequence, Item) line: 171
ElementConstructor(AbstractExpression).eval(Sequence) line: 43
PathExpr.eval(Sequence, Item) line: 159
PathExpr(AbstractExpression).eval(Sequence) line: 43
PathExpr.eval(Sequence, Item) line: 159
PathExpr(AbstractExpression).eval(Sequence) line: 43
LetExpr.eval(Sequence, Item, Sequence) line: 128
LetExpr(BindingExpression).eval(Sequence, Item) line: 110
LetExpr(AbstractExpression).eval(Sequence) line: 43
PathExpr.eval(Sequence, Item) line: 159
PathExpr(AbstractExpression).eval(Sequence) line: 43
XQuery.execute(CompiledXQuery, Sequence) line: 136
LocalXPathQueryService.execute(String[], NodeSet, CompiledExpression, String) line: 330
LocalXPathQueryService.doQuery(String, String[], NodeSet, String) line: 276
LocalXPathQueryService.query(String, String) line: 112
LocalXPathQueryService.query(String) line: 103
CopyOfXPathQueryTest.queryAndAssert(XQueryService, String, int, String) line: 147
CopyOfXPathQueryTest.testConvertToBoolean() line: 201
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
Method.invoke(Object, Object...) line: 585
CopyOfXPathQueryTest(TestCase).runTest() line: 154
CopyOfXPathQueryTest(TestCase).runBare() line: 127
TestResult$1.protect() line: 106
TestResult.runProtected(Test, Protectable) line: 124
TestResult.run(TestCase) line: 109
CopyOfXPathQueryTest(TestCase).run(TestResult) line: 118
TestSuite.runTest(Test, TestResult) line: 208
TestSuite.run(TestResult) line: 203
RemoteTestRunner.runTests(String[], String) line: 478
RemoteTestRunner.run() line: 344
RemoteTestRunner.main(String[]) line: 196

value == Double.NaN
is false !!!!!!!!!!!!!!!!! is value is a double equal to NaN .


$ grep --recursive xpath-functions src/
src/org/exist/xquery/Function.java:             "http://www.w3.org/2003/05/xpath-functions";
src/org/exist/xquery/functions/ModuleImpl.java: public final static String NAMESPACE_URI = "http://www.w3.org/2003/05/xpath-functions";
src/org/exist/xquery/Module.java:               "http://www.w3.org/2003/05/xpath-functions";
src/org/exist/util/Collations.java:    public final static String CODEPOINT = "http://www.w3.org/2004/07/xpath-functions/collation/codepoint";

Unbound prefixes still not solved !!!!!!!!!!!!!!!!!!!

I debugged :
/abc:*
when the prefix abc is not bound. Currently it behaves like /* .
A VirtualNodeSet is created
- this is normal
- shall we throw an error later when the  VirtualNodeSet is expanded ?

In fact at http://www.w3.org/TR/xquery/#node-tests I got the answer about what to do :
A node test can have the form NCName:*. In this case, the prefix is expanded in the same way as with a QName, using the statically known namespaces in the static context. If the prefix is not found in the statically known namespaces, a static error is raised [err:XPST0008].

So in XQueryContext there should be a way to check if a prefix has been declared. When creating a NameTest in XQueryTreeParser the check can be done and the exception [err:XPST0008] throwed .

2005-09-26 : still not solved !!!!!!!!!!!!!!!!!!!


Tried to solve
XPathQueryTest.bugtestStarAxisConstraints()
but couldn't .
query:
/ * [ ./ * / t:title ]



J Revision Log (Non-Normative) - in W3C Xquery specification

JMV: the red items are to check. The greens are done . I leave aside the Schema features.

This log records the changes that have been made to this document since the Last Call Draft of 12 November 2003.

J.1 23 July 2004

  1. An ordering declaration has been added to the Prolog, which affects the ordering semantics of path expressions, FLWOR expressions, and union, intersect, and except expressions. In addition, ordered and unordered operators have been introduced that permit ordering semantics to be controlled at the expression level within a query.

  2. The SequenceType syntax has been simplified. SchemaContextPath is no longer part of the SequenceType syntax. Also, the section on SequenceType Matching has been rewritten to make its terminology more consistent with the Formal Semantics document.

  3. Validation has been separated from construction. Validation now occurs only as a result of an explicit validate expression. Validation modes are strict and lax, and are specified on the validate expression. New construction modes strip and preserve have been defined and are declared in the Prolog. The notion of "validation context" has been deleted. The XQuery definition of validation has been converged with the definition used in XSLT.

  4. It is now permitted for multiple user-defined functions to have the same name as long as they have different numbers of arguments.

  5. The term xdt:untypedAny is changed to xdt:untyped.

  6. xs:anyType is no longer an abstract type, but is now used to denote the type of a partially validated element node. Since there is no longer a meaningful distinction between abstract types and concrete types, these terms are no longer used in this document.

  7. Value comparisons now return () if either operand is ().

  8. Computed namespace constructors are now completely static and are allowed only inside a computed element constructor. Namespace declarations in a computed element constructor must come before the element content, and must consist entirely of literals. The namespace prefix is optional. If absent, it has the effect of setting the default namespace for elements and types within the scope of the constructed element.

  9. The name of the Predicates grammar production has been changed to PredicateList.

  10. Grammar changes have been made in the Prolog, including constraints on the order of declarations and imports.

  11. The grammar production formerly called a FilterStep is now called a FilterExpression and is described in a separate section rather than as part of a path expression.

  12. The precedence of the cast and treat operators in the grammar has been increased.

  13. The precedence of unary arithmetic operators has been increased.

  14. The syntax for variable initialization in the Prolog now uses an assignment operator (":="). Also, circularities in variable initialization are now considered to be a static errors.

  15. The appendix containing a list of error messages has been made non-normative.

  16. An error is raised if a module attempts to import itself (target namespace of importing module and imported modules are the same).

  17. A schema can now be imported without specifying either a target namespace or a location hint.

  18. Module imports and schema imports now accept multiple location hints, representing multiple physical resources in the same module or schema.

  19. CData Sections are no longer considered to be constructors, but are simply a notational convenience for embedding special characters in the content of an element or attribute constructor.

  20. Three new components have been added to the static context: XQuery Flagger status, XQuery Static Flagger status, and context item static type. (Note: Flagger status items were later deleted.)

  21. The expression data(()) no longer raises a static error, although its static type is empty.

  22. If a dynamic implementation detects a static error, the dynamic evaluation phase is optional.

  23. XPath allows host languages to specify whether they recognize the concept of a namespace node. XQuery does not recognize namespace nodes. Instead, it recognizes an "in-scope namespaces" property of an element node.

  24. The specification now clearly distinguishes between "statically-known namespaces" (a static property of an expression) and "in-scope namespaces" (a dynamic property of an element).

  25. An order by clause may now accept values of mixed type if they have a common type that is reachable by numeric type promotion and/or moving up the type derivation hierarchy, and if this common type has a gt operator.

  26. In element and document node constructors, if the content sequence contains a document node, that node is replaced by its children (this was previously treated as an error).

  27. Atomization now applies to the name expression of a computed processing instruction constructor.

  28. It is now implementation-defined whether undeclaration of namespace prefixes in an element constructor is supported. If supported, this feature conforms to the semantics of Namespaces 1.1. In other words, if an element constructor binds a namespace prefix to the zero-length string, any binding of that prefix defined at an outer level is suspended within the scope of the constructed element.

  29. The names of certain components in the static and dynamic contexts have been changed to make them more consistent and descriptive.

  30. In a computed text node constructor, the expression enclosed in curly braces is no longer optional, since it is not possible to construct an empty text node.

  31. The definition of the idiv operator has been made consistent with the Functions and Operators document. The idiv operator accepts operands of any numeric types, performs a division, and truncates the result to an integer.

  32. Certain grammar productions have been renamed and reordered to make the grammar easier to read.

  33. The section on constructor functions has been edited to make it more consistent with the definition of constructor functions in the Functions and Operators document.

  34. The Comments section has been moved out of the Primary Expressions section, because a comment is not a primary expression.

  35. Rules for processing comment constructors have changed, to ensure that the resulting comment does not contain adjacent hyphens or end with a hyphen.

  36. Some additional constraints have been added to the section titled "Consistency Constraints".

  37. Various minor changes, both editorial and substantive, have been made in response to public comments and working group discussions.

J.2 29 Oct 2004

  1. The last step in a path expression is now allowed to return a sequence of atomic values or a sequence of nodes (mixed nodes and atomic values are not allowed.)

  2. A value of type xs:QName is now defined to consist of a "triple": a namespace prefix, a namespace URI, and a local name. Including the prefix as part of the QName value makes it possible to cast any QName into a string when needed.

  3. Local namespace declarations have been deleted from computed element constructors. No namespace bindings may be declared by a computed element constructors. Also, the body of a computed element constructor has changed from ExprSingle to Expr.

  4. The Prolog has been reorganized into three parts which must appear in this order: (a) Setters; (b) Namespace declarations and module and schema imports; (c) function and variable declarations.

  5. A new "inherit-namespaces" declaration has been added to the Prolog, and "namespace inheritance mode" has been added to the static context.

  6. An "encoding" subclause has been added to the Version Declaration in the Prolog.

  7. A new declaration has been added to the Prolog to control the query-wide default handling of empty sequences in ordering keys ("empty greatest" or "empty least".) "Default order for empty sequence" has been added to static context.

  8. XQuery Processing Model figure has been revised to include a reference to modules.

  9. The expression in the where-clause of a FLWOR expression has changed from Expr to ExprSingle.

  10. Clarification: The same variable name may be bound in more than one for or let clause in a given FLWOR expression. This is not an error, but each new binding occludes the previous one, which is no longer accessible in the rest of the FLWOR expression.

  11. In the static context, "current date" and "current time" have been replaced by "current dateTime", which is defined to include a timezone. Also, an implementation is required to initialize the current dateTime and implicit timezone.

  12. Computed comment constructors now raise an error rather than trying to "fix up" an invalid comment by inserting blanks.

  13. The div operator can now be used to divide two yearMonthDurations or two dayTimeDurations. In either case, the result is of type xs:decimal.

  14. Additional signatures have been added to the fn:error function.

  15. The to operator now accepts an empty sequence as an operand, and returns empty sequence.

  16. Support for XML 1.1 and Namespaces 1.1 have been bundled together and defined as an optional feature. Various aspects of query processing and serialization that depend on this optional feature have been identified.

  17. Descriptions of the arithmetic and logical operators have been edited to eliminate dependencies on the order of operand evaluation.

  18. The term "module resource," originally defined as a fragment of XQuery code that can independently undergo static analysis, has been replaced by the term "module."

  19. Cyclic module imports are no longer permitted. A module M may not import another module that directly or indirectly imports module M.

  20. A new zero-argument version of fn:collection has been defined, to provide access to an implementation-dependent default sequence.

  21. A Mime encoding (application/xquery) has been defined for XQuery, and a new Appendix has been added to describe it.

  22. The operator mapping for numeric ge and le operators has been changed.

  23. The section on typed values and string values has been edited to clarify that an implementation may store either value and derive the other. Also, an example has been added to illustrate the typed value of a union type.

  24. In a validate expression, the node to be validated is now mapped directly to an Infoset using the mapping defined in the Data Model, rather than being serialized and parsed.

  25. Various minor changes, both editorial and substantive, have been made in response to public comments and working group discussions.

J.3 7 January 2005

  1. A new section on Conformance has been added to the document.

  2. The definition of Effective Boolean Value has changed.

  3. The syntax for Pragmas has changed. The new syntax is described in two new sections titled Extension Expressions and Option Declaration.

  4. The XMLSpace declaration is now called a Boundary-space declaration, to distinguish it from the unrelated attribute named xml:space.

  5. Changes have been made to the rules for namespace bindings in element nodes that are copied by an element constructor. In the grammar, "declare inherit-namespaces" has changed to "declare copy-namespaces" with some new options.

  6. Values of type xs:anyURI are now promotable to the type xs:string, and are compared and ordered using the same default collation used for strings.

  7. A new symbol URILiteral has been added to the grammar and used in places where a statically-known URI is required.

  8. In a function declaration, if the declared function name has no namespace prefix, it is now considered to be in the default namespace for its module.

  9. The XQuery Flagger and Static Flagger have been deleted from the specification.

  10. Constructor functions are now defined for types that are in no namespace.

  11. Type-related terms such as "schema type", "sequence type", "static type", "dynamic type", and "type annotation" have been given better definitions and more consistent usage.

  12. Several error codes have been redefined or deleted.

  13. Minor corrections have been made to the Operator Precedence Table.

  14. Various minor changes, both editorial and substantive, have been made in response to public comments and working group discussions.

J.4 21 February 2005

  1. Error codes have been expanded from six characters to eight characters. Several error codes have been added, deleted, or modified.

  2. New rules and examples have been added to the section on Errors and Optimization.

  3. A new consistency constraint has been added: the value of the context item must match its static type.

  4. Text has been added clarifying that, if the Static Typing Feature is not in effect, expressions have an implementation-dependent static type.

  5. A rule has been added requiring an implementation to perform whitespace normalization on URILiterals.

  6. The grammatical rules governing the required order of items in the Prolog have been changed.

  7. The rule prohibiting a module from importing its own namespace has been deleted.

  8. The description of the Validate Expression has been reorganized and rewritten.

  9. Rules governing the baseURI properties of nodes created or copied by constructors have changed.

  10. The References section has been reorganized.

  11. A new appendix has been added, enumerating all the implementation-defined items.

  12. Various minor changes, both editorial and substantive, have been made in response to public comments and working group discussions.

J.5 04 April 2005

  1. The keyword empty has been changed to void in the SequenceType grammar, in order to avoid conflict with the built-in function named empty.

  2. Unordered mode now applies to sequences of atomic values generated by the last step of a path expression, as well as node sequences.

  3. The section on constructor functions has been rewritten. The semantics of constructor functions are now defined as equivalent to a cast expression.

  4. The definition of boundary whitespace has been revised.

  5. The section on validate expressions now specifies that the default validation mode is strict (this default was inadvertently omitted from earlier drafts.)

  6. The version of Unicode that is supported by an XQuery implementation is now implementation-defined.

  7. The format in which errors are reported by an XQuery implementation is now implementation-defined.

  8. Various minor changes, both editorial and substantive, have been made in response to public comments and working group discussions.

J Revision Log (Non-Normative)

J.1 7 July 2005

  1. An error has been corrected in the definition of the expansion of leading-slash in a path expression.

  2. Operators eq and ne are now defined for the xs:duration type.

  3. Replaced reference to RFC2396 (URI's) with references to RFC3986 and RFC3987 (IRI's). Inserted text indicating that IRI's are accepted where URI's are expected. Also replaced obsolete reference to RFC1738 by an updated reference to RFC3987.

  4. Added text to section 2.5.2 clarifying that if the nilled property of an element node is true, its typed value is the empty sequence.

  5. Added a consistency constraint stating that the "xml" prefix is predefined, and it cannot be bound to anything else, and no other prefix can be bound to the same URI.

  6. In Section 3.7.1.3, if construction mode is strip (Rule 1-e-ii-C-II), when stripping the type of an attribute: If the name of the attribute is xml:ID, then the "is-ID" property of the attribute is set to True; otherwise False.

  7. Changed Section 3.7.1.3, Rule 1-e-ii-E. The base-URI property of a copied node is no longer preserved, but inherited from the new parent. Aligns XQuery with XSLT. New text for rule: "When an element or processing instruction node is copied, its base-uri property is set to be the same as that of its new parent, with the following exception: if a copied element node has an xml:base attribute, its base-uri property is set to the value of that attribute, resolved (if it is relative) against the base-uri property of the new parent node. All other properties of the copied nodes are preserved."

  8. In Section 3.7.1.3, rules for copying element nodes when construction mode is preserve and copy-namespaces mode is no-preserve have been changed to introduce a new type error: XQTY0086.

  9. The string value of an empty direct constructor is now defined as a zero-length string.

  10. Some changes have been made in the default values of serialization parameters.

  11. Removed from the main part of the document any references to line-ending normalization. Affects 3.1.1 (Literals) and 3.7.1.3 (Dir. Elem. Constructor--Content, Rule 1a). Responds to Bug 1307. Line ending normalization will be applied globally by the parser rather than by individual expressions.

  12. In SequenceType syntax, keyword void is changed to empty-sequence.

  13. Changes to function names: Changes function names: fn:subtract-dateTimes-yielding-dayTimeDuration becomes op:subtract-dateTimes and fn:subtract-dates-yielding-dayTimeDuration becomes op:subtract-dates (affects operator mapping table).


J.2 15 September 2005

  1. An error code (XQST0087) has been defined for an invalid encoding specification in a Version Declaration.

  2. Rules for processing the content of a direct element constructor have been edited to clarify that boundary whitespace is processed (possibly stripped) before entity and character references are expanded.

  3. Rules for constructing attributes with the name xml:id have been modified. Additional normalization rules apply to these attributes. The error (XQST0082) that was raised when the value of an xml:id attribute was not a valid NCName has been deleted.

  4. Error XPDY0044 is clarified to apply whenever the node-name property of a constructed attribute node is xmlns or xmlns:*, regardless of how the node-name was computed.

  5. The descriptions of all the node constructors have been changed to specify that all newly-constructed nodes have an empty parent property. The parent property of the new node can be set by an outer-level constructor (if any).

  6. The definition of a range expression has been edited to clarify that, when the lower and upper bound of a range are the same integer, only a single integer is returned by the range expression.

  7. Values of numeric literals are now defined by the rules for casting from xdt:untypedAtomic into the appropriate numeric type. This change defines overflow and underflow behavior for numeric literals.


2005-09-17

Wolfgang corrected this :
declare variable $v as xs:string external;
this erased the value set by XQueryGenerator.


2005-09-16

http://cocoondev.org/main/117/42.html another CMS


giulio also jeanMarc there is http://www.bluexml.com/content/portal?cocoon-portal-action=2&cocoon-portal-event=0 it use eXist as backend

wolf77 hmmm, it seems bluexml has matured since I checked it last time ;-)



Suggestions:
    jmvanel    I can paste the two main error stacks .

Are you aware of this project :
http://www.informatik.hu-berlin.de/mac/xquery/
The download is :-( not accessible .



A question about XQuery :
<a>{ $x, $y }</a>
and
<a>{ ( $x, $y ) }</a>
both work on eXist; are they both conforming to the spec ?

wolf77_ I think yes, moment ...

wolf77_ Yes, the EBNF defines EnclosedExpr as: "{" Expr "}" and Expr is defined as: ExprSingle ("," ExprSingle)*

jmvanel Maybe is it good practice to use :
<a>{ ( $x, $y ) }</a>
to prepare the human reader for the following commas .

jmvanel It the cost is the same ...

wolf77_ yes, the cost should be the same.

I was surprised that  :
let $w := <w initial="val" />
return <a>{ $w/@initial }</a>
returns
        <a initial="val"/>
instead of:
        <a>val</a>
but Saxon says the same .


* Would that be difficult to implement the abiliy to use an old eXist server with the newest client ?
I can paste the two main error stacks .

wolf77 Yes, ok. Show me the stack traces...

jmvanel Caused by: org.apache.xmlrpc.XmlRpcException:org.exist.EXistException:position parameter out of bounds in call to retrieve: 0
at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java:457)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:163)
at org.exist.xmldb.RemoteXMLResource.getContent(RemoteXMLResource.java:158)
at org.exist.client.QueryDialog$QueryThread.run(QueryDialog.java:416)

Caused by: org.apache.xmlrpc.XmlRpcException:org.exist.EXistException:result set unknown or timed out
at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java:457)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:163)
at org.exist.xmldb.RemoteXMLResource.getContent(RemoteXMLResource.java:158)
at org.exist.client.QueryDialog$QueryThread.run(QueryDialog.java:416)

jmvanel The second one appears in loop after the first.

wolf77 To fix the first one, change line 155 in org.exist.xmldb.RemoteXMLResourceto

wolf77 params.addElement(newInteger(pos + 1));


2005-08-01

Refactoring

Remains to do :

Guessing last modification time for an XQuery result

     *  the HTTP header Last-Modified is filled with most recent time stamp among all
     *  XQuery documents appearing in the actual response.
     *  Note however, that the actual response can be influenced, through tests in the query,
     *  by documents more recent.
wget --output-document=- --save-headers \
'http://zamia:8080/exist/servlet/db/?_query=//property'

2005-07-31

Refactoring

Name changes (See ContentLoadingObserver) :
addRow() --> markNode()
storeElement() --> markElement()

The relevant stack :
Thread [Thread-5] (Suspended)
NativeElementIndex.addRow(QName, NodeProxy) line: 99
NativeBroker.endElement(NodeImpl, NodePath, String) line: 2672
Indexer.endElement(String, String, String) line: 277
SAXParserImpl$JAXPSAXParser(AbstractSAXParser).endElement(QName, Augmentations) line: not available
XMLNSDocumentScannerImpl.scanEndElement() line: not available
XMLNSDocumentScannerImpl$NSContentDispatcher(XMLDocumentFragmentScannerImpl$FragmentContentDispatcher).dispatch(boolean) line: not available
XMLNSDocumentScannerImpl(XMLDocumentFragmentScannerImpl).scanDocument(boolean) line: not available
XIncludeAwareParserConfiguration(XML11Configuration).parse(boolean) line: not available
XIncludeAwareParserConfiguration(XML11Configuration).parse(XMLInputSource) line: not available
SAXParserImpl$JAXPSAXParser(XMLParser).parse(XMLInputSource) line: not available
SAXParserImpl$JAXPSAXParser(AbstractSAXParser).parse(InputSource) line: not available
SAXParserImpl$JAXPSAXParser.parse(InputSource) line: not available
Collection.store(Txn, DBBroker, IndexInfo, InputSource, boolean) line: 771
LocalCollection.storeXMLResource(LocalXMLResource) line: 642
LocalCollection.storeResource(Resource, Date, Date) line: 556
LocalCollection.storeResource(Resource) line: 548
InteractiveClient.parse(File[], UploadDialog) line: 1426
ClientFrame$37.run() line: 967

The sequence of calls in NativeBroker during  storage of a simple XML document :
<a attr="aaa"><b>bbb</b></a>
Called (indirectly) by SAX parser :
    doIndex()                 case Node.ELEMENT_NODE        currentPath == /a
    doIndex()                 case Node.ATTRIBUTE_NODE
    doIndex()                 case Node.ELEMENT_NODE        currentPath == /a/b
    doIndex()                 case Node.TEXT_NODE
    endElement()     
currentPath == /a/b      content == null
    endElement()      currentPath == /a        content == null
Called by Collection.store() :
            broker.storeDocument(transaction, document);
            broker.closeDocument();
            broker.flush();

Note: in Indexer.endElement() we have this dependancy on indexers :
        XMLString elemContent = null;
if (GeneralRangeIndexSpec.hasQNameOrValueIndex(last.getIndexType())) {
elemContent = (XMLString) nodeContentStack.pop();
}
The above sequence of calls correspond to SAX events; the callback functions for the indexers (interface ContentLoadingObserver ) could bear the same names as the SAX events. This will make things clear for developpers. However ContentLoadingObserver.storeAttribute() has no corresponding method in SAX.

Call stack  when removing a Document :
Thread [Thread-7] (Suspended)
NativeValueIndexByQName(NativeValueIndex).dropIndex(DocumentImpl) line: 336
NativeValueIndexByQName.dropIndex(DocumentImpl) line: 298
NativeBroker.notifyDropIndex(DocumentImpl) line: 339
NativeBroker.removeDocument(Txn, DocumentImpl, boolean) line: 2030
NativeBroker(DBBroker).removeDocument(Txn, DocumentImpl) line: 418
Collection.removeDocument(Txn, DBBroker, String) line: 677
LocalCollection.removeResource(Resource) line: 518
ClientFrame$33.run() line: 796
Thread.run() line: 595


Processing XML with Java -
Welcome to Processing XML with Java , a complete tutorial about writing Java ...
It contains over 1000 pages of detailed information on SAX, DOM, JDOM,

2005-07-30

Discussion with Perig

New features must be in permanent location.
I proposed this new Wiki page for rather "large" features, with several days' work .
    Otherwise the pages at sf.net are to be used .

For example, in DBBroker (the *abstract* class for brokers), dataDir is configured...
    Perig_    It should be configured in *native*Broker IMHO (because Native = Local File system).

A classical Article about Spring and others : "Inversion of Control Containers and the Dependency Injection pattern"
http://martinfowler.com/articles/injection.html

Nice thing is that there is allready an integration of XML:DB data sources in Spring :
http://Springxmldb.sf.net
    But this is using eXist with Spring from the outside . I think about using the Spring facilities inside eXist .

Pas facile : comme cela a été signalé dans la liste, la notion de "journal" s'emmélait joyeusement avec celle de "log".
    A propos, truc totalement cosmétique :
    DatableImpl, ligne 127 :
    DatabaseImpl
    System.out.println("configuring " + dbName + " using " + home + '/' + file);
    Remplacer ce "/" par File.pathSeparator
    Gaffe d'ailleurs : c'est souvent pathSeparatorChar qui est utilisé : les lecteurs réseau de Windows risquent de ne pas aimer.

Guessing last modification time for an XQuery result

Is there some places in code where one can be pretty shure that a database node has been really used in the ouput of the query ?
    wolf77    Just at the very end, before the query result is returned.
    org/exist/xquery/XQuery.java

So in org/exist/xquery/XQuery.java
in theory I could do a recursive loop on
Sequence result
to examine each node's Document, but that might be expensive .
Either it would be an option, or above say, 100 nodes, do not output the last modification time .
    wolf77    yes, that might be a good solution. But you could just see how much performance it costs and then decide.

Recovery bugs

I have to re- read
http://wiki.exist-db.org/space/LoggingAndRecovery
and review the code .
I begin a comment on
LoggingAndRecovery .

jmv@zamia: /usr/src/usr2/eXist-1.0 $ grep --recursive writeToLog src | wc
     79     256    7316

Refactoring

What is planned


finish the refactoring in NativeBroker with interface ContentLoadingObserver

refactor NativeValueIndex (900 lines) to make it look more like
NativeValueIndexByQName .
And remove calls deprecated from Indexable :
public byte[] serialize(short collectionId, boolean caseSensitive);
to be replaced by :
byte[] serializeValue (int offset, boolean caseSensitive);

Doing it


Wolf ?

Still not modifying NativeBroker ?

Before calling valueIndex.storeAttribute()
in NativeBroker.NodeProcessor.doIndex(),
currentPath.addComponent()is *not* called

and before calling qnameValueIndex.storeAttribute()
currentPath.addComponent()*is* called
What's the use of this call ? probably obsolete ?

Began moving code from NativeBroker to NativeValueIndexByQName

Before calling valueIndex.storeAttribute()
in NativeBroker.NodeProcessor.doIndex(),
currentPath.addComponent() is *not* called

and before calling qnameValueIndex.storeAttribute()
currentPath.addComponent() *is* called
What's the use of this call ? probably obsolete ?
    |<--    Perig has left irc.freenode.net (Read error: 54 (Connection reset by peer))
    jmvanel   
Began moving code from NativeBroker to NativeValueIndexByQName
I'll commit , and then extract more code

Later I'll extract inner class NodeProcessor /
    -->|    Perig (~chatzilla@vil35-2-82-236-174-106.fbx.proxad.net) has joined #existdb
    |wolf77|    Calling currentPath.addComponent() is certainly wrong. Either it has to be called for all indexes or never. How does this look in the original code?
    jmvanel    Let see in the past ...
    Just before I introduced NativeValueIndexByQName
    That is, may 25
    revisoin 141
    |wolf77|    ok, in that revision, addComponent() occurred at the very beginning of the ATTRIBUTE case in method store().
    that makes sense. So with the current code, searches for attributes should *fail*! Is this covered by the tests ...
    No, it's not covered by ValueIndexTest. Could you check that?
    jmvanel    Still trying to understand what this call does ...
    OK :
    |wolf77|    It's simple: in order to check if a node has to be added to an index, we need to compare the entire path leading to the node with the definitions in the configuration. NodePath is the class for storing the path.
    jmvanel    currentPath passed in argument doesn't include the current node , right ?
    |wolf77|    it just contains the path up to the parent element, yes.

There *is* a test at line 113 in ValueIndexTest : //item[@xx:test = 123]
    But even if the value index is KO for the attributes , the tests passes .
    |wolf77|    yes, correct. However, we don't know if the expression does really use the value index. Have to look at the test output.
    jmvanel    Bacause the ValueIndex is not testable .
    If the the value index is OK the only difference is that the test is quicker . Right ?
    |wolf77|    Right. No, the log output doesn't show it's using the index.


In ValueIndexTest vI get the message
 The specified type: xs:double ... does not implement interface Indexable.

Concerning this attribute :
        <price specialprice="true">1024.99</price>

With this thread :

Thread [main] (Suspended (breakpoint at line 723 in NativeValueIndex))
    NativeValueIndex.convertToAtomic(int, String) line: 723
    NativeValueIndex.storeAttribute(RangeIndexSpec, AttrImpl) line: 143
    NativeBroker$NodeProcessor.doIndex() line: 993
    NativeBroker.store(Txn, NodeImpl, NodePath, boolean) line: 2598
    NativeBroker(DBBroker).store(Txn, NodeImpl, NodePath) line: 470
    Indexer.startElement(String, String, String, Attributes) line: 561
    SAXParserImpl$JAXPSAXParser(AbstractSAXParser).startElement(QName, XMLAttributes, Augmentations) line: not available
    XMLNSDocumentScannerImpl.scanStartElement() line: not available
    XMLNSDocumentScannerImpl$NSContentDispatcher(XMLDocumentFragmentScannerImpl$FragmentContentDispatcher).dispatch(boolean) line: not available
    XMLNSDocumentScannerImpl(XMLDocumentFragmentScannerImpl).scanDocument(boolean) line: not available
    XIncludeAwareParserConfiguration(XML11Configuration).parse(boolean) line: not available
    XIncludeAwareParserConfiguration(XML11Configuration).parse(XMLInputSource) line: not available
    SAXParserImpl$JAXPSAXParser(XMLParser).parse(XMLInputSource) line: not available
    SAXParserImpl$JAXPSAXParser(AbstractSAXParser).parse(InputSource) line: not available
    SAXParserImpl$JAXPSAXParser.parse(InputSource) line: not available
    Collection.store(Txn, DBBroker, IndexInfo, InputSource, boolean) line: 771
    LocalCollection.storeXMLResource(LocalXMLResource) line: 642
    LocalCollection.storeResource(Resource, Date, Date) line: 556
    LocalCollection.storeResource(Resource) line: 548
    ValueIndexTest.storeXMLFileAndGetQueryService(String, String) line: 177
    ValueIndexTest.testStrings() line: 100
    NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
    NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
    DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
    Method.invoke(Object, Object...) line: 585
    ValueIndexTest(TestCase).runTest() line: 154
    ValueIndexTest(TestCase).runBare() line: 127
    TestResult$1.protect() line: 106
    TestResult.runProtected(Test, Protectable) line: 124
    TestResult.run(TestCase) line: 109
    ValueIndexTest(TestCase).run(TestResult) line: 118
    TestSuite.runTest(Test, TestResult) line: 208
    TestSuite.run(TestResult) line: 203
    RemoteTestRunner.runTests(String[], String) line: 478
    RemoteTestRunner.run() line: 344
    RemoteTestRunner.main(String[]) line: 196

As Wolfgang wrote, I have to call
addComponent() at the very beginning of the ATTRIBUTE case in method doIndex(). It works .
I pass build.sh and I'll commit (19:37).

2005-07-29

Working on my client's xquery .

Added in class LocationPath a primitive XQuery profiler ; to enable it , change :
profilingEnabled
= true ;
in org.exist.xquery/LocationStep.java


New XQuery function to return a date-valued HTTP header : example :

let $datetime := '2005-07-22T22:22:22'
let $dummy := request:set-date-header( 'Last-Modified', xs:dateTime($datetime) )
return $datetime

2005-07-28

While working on my client's xquery, I'm thinking of ways of attracting new users.
I thought that eXist must have a good stomach, i.e. able to absorb many different foods.
I just added HTML input. Zip would not be difficult to add, it is already on the "catalog" of possible new new features.
There is an important type of data, that is not suitable for relational databases: mail . If I could find an XML parser for mailbox, it would be nice to integrate it to eXist; But I can't find one ...
    wolf77    hmmm, maybe through a text2xml parser? I think I read about one ...
    DiZzZz    mailbox 2 xml..... hmmm there a at least 2 formats. mbox and ....... the other one
    jmvanel    I found no javaCC grammar for mail . What is name of the concurrent to javaCC ?
    wolf77    antlr?
    jmvanel    Yes.
    Nothing for antlr .
    Maybe there there is something to reuse from javamail .
    jmvanel    There are "local store providers" for JavaMail, supporting different Mail formats, e.g. mstor.sf.net for mbox , or javamaildir.sf.net for qmail format .

2005-07-27

2005-07-25

Intesresting disussion with Perig . To really use eXist he would need a spatial indexing.
http://www.geotools.org/Getting+Started

I see two solutions :
    - develop a new special index mapping coordinates to nodes,
    and integrate geographic algo. to search the nearest points
    - integrate XQuark, and delegate the GIS part to PostgreSQL or others
  but there is some work in XQuark to pass the GIS SQL queries to the XQuery level

Added
http://wiki.exist-db.org/space/Possible+new+features:+a+catalog



system:ft-index-lookup(node(), xs:string)
    A query like: /dblp/inproceedings[title &= "sozial*"]
    can be reformulated in different ways to use the system:ft-index-lookup() function. The most efficient formulation is:
    would be: /dblp/(system:ft-index-lookup(ancestor::title, "sozial*")/parent::inproceedings)
    twu    This is called "Predicate Push Down"?
    wolf77    We don't have a name for it yet ;-) "Selection Pull Up"?
    twu    I read about this technique here: http://www.ispras.ru/~grinev/mypapers/phd-short.pdf

Attribute push down is one technique to reduce element construction (which is costly).
    The author also states that inlining of user defined function is important, since only then the whole query may be optimized.


Wolfgang , for the create index implementation,
    we could have an iterator that would provide the same events to NativeBroker as the document loader, but by iterating on the DB or a collection .



wolf77 Pooh, I finally merged my NativeBroker with your changes.

2005-07-24


the names of 4 classes that read documents , and that could update a field latestTimeStamp in the Context

to output that in the HTTP header Last-Modified .


wolf77 FunDoc, ExtCollection, ExtDocument, RootNode

As requested, I started to write up some design notes here: http://wiki.exist-db.org/space/LoggingAndRecovery


I implemented the last modification time column in the GUI : modifed classes :

2005-07-23

Time 10:54

I updated my working dir. with Piotr's date code.

Need to commit the .classpath :
the Xerces files jar files have changed after Piotr's date code change.

I got rid of this frustrating little bug that is was here for weeks and nobody cared . Out of 183 tests, just one failure !
    [junit] org.exist.xquery.XPathException: cannot compare numeric value to non-numeric value
[junit] at org.exist.xquery.value.NumericValue.compareTo(NumericValue.java:74)
[junit] at org.exist.xquery.functions.FunDeepEqual.deepEquals(FunDeepEqual.java:72)
[junit] at org.exist.xquery.functions.FunDeepEqual.eval(FunDeepEqual.java:63)
[junit] at org.exist.xquery.PathExpr.eval(PathExpr.java:144)
[junit] at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:43)
[junit] at org.exist.xquery.XQuery.execute(XQuery.java:135)
[junit] at org.exist.xmldb.LocalXPathQueryService.execute(LocalXPathQueryService.java:330)
[junit] at org.exist.xmldb.LocalXPathQueryService.doQuery(LocalXPathQueryService.java:276)
[junit] at org.exist.xmldb.LocalXPathQueryService.query(LocalXPathQueryService.java:112)
[junit] at org.exist.xmldb.LocalXPathQueryService.query(LocalXPathQueryService.java:103)
[junit] at org.exist.xquery.test.DeepEqualTest.assertQuery(DeepEqualTest.java:248)
[junit] at org.exist.xquery.test.DeepEqualTest.testAtomic5(DeepEqualTest.java:50)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:585)
[junit] at junit.framework.TestCase.runTest(TestCase.java:154)
[junit] at junit.framework.TestCase.runBare(TestCase.java:127)
[junit] at junit.framework.TestResult$1.protect(TestResult.java:106)
[junit] at junit.framework.TestResult.runProtected(TestResult.java:124)
[junit] at junit.framework.TestResult.run(TestResult.java:109)
[junit] at junit.framework.TestCase.run(TestCase.java:118)
[junit] at junit.framework.TestSuite.runTest(TestSuite.java:208)
[junit] at junit.framework.TestSuite.run(TestSuite.java:203)
[junit] at junit.framework.TestSuite.runTest(TestSuite.java:208)
[junit] at junit.framework.TestSuite.run(TestSuite.java:203)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:326)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:560)
[junit] Loading mime table from file /usr/src/usr2/eXist-1.0/mime-types.xml
corresponding to the XQuery :
deep-equal(42, 'hello')

Inside this test in org.exist.xquery.functions.FunDeepEqual :
if (Type.subTypeOf(a.getType(), Type.ATOMIC) || Type.subTypeOf(b.getType(), Type.ATOMIC)) {
I replaced this
if (!Type.subTypeOf(a.getType(), Type.ATOMIC) || 
!Type.subTypeOf(b.getType(), Type.ATOMIC)) return false;
with this line , simpler and more general :
if ( a.getType() != b.getType() ) return false;


Time 15:05

It was not as simple as we thought ! First, the original logical test was correct .
But the function deep-equal() should not throw an error when atomic types are not compatible. The spec (http://www.w3.org/TR/xpath-functions/#func-deep-equal) doesn't say so. Thus the function deep-equal() has a very different behavior compared to operator eq , which must report an error (http://www.w3.org/TR/xquery/#id-comparisons) :

If the types of the operands, after evaluation, are not a valid combination for the given operator, according to the rules in B.2 Operator Mapping, a type error is raised [err:XPTY0004].

Trying with Saxon 8.4 from Saxonica

$ echo '111 eq "gggg"' > /tmp/bb.xq
$ xquery /tmp/bb.xq
Error
XPTY0004: Cannot compare xs:integer to xs:string
Failed to compile query

$ echo 'fn:deep-equal(111, "gggg")' > /tmp/bb.xq
$ xquery -wrap /tmp/bb.xq
<?xml version="1.0" encoding="UTF-8"?>
<result:sequence xmlns:result="http://saxon.sf.net/xquery-results" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<result:atomic-value xsi:type="xs:boolean">false</result:atomic-value>

Thus it's not good to call the algo. that is being used for the eq operator :
        return GeneralComparison.compareAtomic( context.getDefaultCollator(),
(AtomicValue)a, (AtomicValue)b,
context.isBackwardsCompatible(), Constants.TRUNC_NONE, Constants.EQ );
Is there some other place in eXist where a tolerant comparison is implemeted or must I implement it from scratch ????

The Saxon mailing list:
http://sourceforge.net/mailarchive/forum.php?forum=saxon-help
No matches found for deep-equal

Time 15:57 :

With this implementation :
        return a.atomize().compareTo(context.getDefaultCollator(), b.atomize()) == 0;

the new test atomic6 crashes :
                assertQuery(true, "deep-equal( 1. , xs:integer(1) )" );
assertQuery(true, "deep-equal( xs:double(1) , xs:integer(1) )" );
true is the result of Saxon.

But with compareAtomic() all is OK :
        return GeneralComparison.compareAtomic( context.getDefaultCollator(),
(AtomicValue)a, (AtomicValue)b,
context.isBackwardsCompatible(), Constants.TRUNC_NONE, Constants.EQ );
}


Wolgang collected some interesting links to small, but nice apps:
one in German: http://pom.bbaw.de/dwb/

This is nice too: http://www.matrix.msu.edu/cls/index

An older one: http://www.tapor.uvic.ca:8080/cocoon/graves/

Also: http://test.perseus.tufts.edu/hopper/catalog

And last but not least, Orbeon OPS has quite a number of eXist + XQuery examples, among them a blog based on eXist.

jmvanel Are you shure all these people DON'T want to add themselves ?

wolf77 http://www.rowas.cz/ck/index-en.htm

    [junit] java.lang.Exception: Stack trace
    [junit]     at java.lang.Thread.dumpStack(Thread.java:1158)
    [junit]     at org.exist.storage.NativeBroker$9.start(NativeBroker.java:1803)
    [junit]     at org.exist.storage.store.DOMTransaction.run(DOMTransaction.java:56)
    [junit]     at org.exist.storage.NativeBroker.objectWith(NativeBroker.java:1819)
    [junit]     at org.exist.dom.DocumentImpl.getFirstChild(DocumentImpl.java:386)
    [junit]     at org.exist.storage.NativeBroker$11.start(NativeBroker.java:2001)
    [junit]     at org.exist.storage.store.DOMTransaction.run(DOMTransaction.java:56)
    [junit]     at org.exist.storage.NativeBroker.removeCollection(NativeBroker.java:2007)
    [junit]     at org.exist.xmldb.LocalCollectionManagementService.removeCollection(LocalCollectionManagementService.java:130)
    [junit]     at org.exist.xquery.test.DeepEqualTest.setupTestCollection(DeepEqualTest.java:269)
    [junit]     at org.exist.xquery.test.DeepEqualTest.setUp(DeepEqualTest.java:283)
    [junit]     at junit.framework.TestCase.runBare(TestCase.java:125)
    [junit]     at junit.framework.TestResult$1.protect(TestResult.java:106)
    [junit]     at junit.framework.TestResult.runProtected(TestResult.java:124)
    [junit]     at junit.framework.TestResult.run(TestResult.java:109)
    [junit]     at junit.framework.TestCase.run(TestCase.java:118)
    [junit]     at junit.framework.TestSuite.runTest(TestSuite.java:208)
    [junit]     at junit.framework.TestSuite.run(TestSuite.java:203)
    [junit]     at junit.framework.TestSuite.runTest(TestSuite.java:208)
    [junit]     at junit.framework.TestSuite.run(TestSuite.java:203)
    [junit]     at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:326)
    [junit]     at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:560)

2005-07-22

New article on eXist :
http://programming.newsforge.com/article.pl?sid=05/06/29/1429231&from=rss

I merge transaction development in NativeBroker and BrokerPool with my refactorings in NativeBroker.

I found a problem (NPE) .

In NativeBroker line 1673 in method
    public Collection getOrCreateCollection(String name)

        final CollectionCache collectionsCache = pool.getCollectionsCache();

The NativeBroker is asking its BrokerPool for the collectionsCache . But it is null ! So the crash at next line.

The problem lies in the new method :
BrokerPool.initialize()
collectionsCache was was set after the call to createBroker() . If I move the collectionsCache definition just before the call to createBroker() , all the tests pass again .


Here is the interesting stack (i.e. not the stack at crash) :
Thread [main] (Suspended)
NativeBroker.<init>(BrokerPool, Configuration) line: 173
BrokerFactory.getInstance(BrokerPool, Configuration) line: 34
BrokerPool.createBroker() line: 444
BrokerPool.initialize() line: 566
BrokerPool.<init>(String, int, int, Configuration) line: 336
BrokerPool.configure(String, int, int, Configuration) line: 117
BrokerPool.configure(int, int, Configuration) line: 95
DOMIndexerTest.setUp() line: 167
DOMIndexerTest(TestCase).runBare() line: 125
TestResult$1.protect() line: 106
TestResult.runProtected(Test, Protectable) line: 124
TestResult.run(TestCase) line: 109
DOMIndexerTest(TestCase).run(TestResult) line: 118
TestSuite.runTest(Test, TestResult) line: 208
TestSuite.run(TestResult) line: 203
RemoteTestRunner.runTests(String[], String) line: 478
RemoteTestRunner.run() line: 344
RemoteTestRunner.main(String[]) line: 196

Suspect code :
NativeBroker: eclipse underlines that in yellow, because root is not used :
    protected void repair() throws PermissionDeniedException {
Collection root = getCollection(ROOT_COLLECTION);
not shure readOnly is correct


2005-07-21

Questions on transactions

Investigations

Integration of transaction code into CVS HEAD

I copied verbatim all this from Wolfgang's code :
storage/txn/
storage/log/
storage/recovery/
storage/lock/
EXistException.java
I did the minimal modifications here so that it compiles and runs (OK) the current CVS tests :
storage/BrokerPool.java
storage/DBBroker.java
storage/NativeBroker.java


Integration of new transaction code for recovery

Dead lock with test.concurrent.ConcurrentTestBase


Full thread dump Java HotSpot(TM) Client VM (1.5.0_03-b07 mixed mode):

"Thread-29" prio=1 tid=0x08324810 nid=0x5c77 waiting for monitor entry [0x8d01f000..0x8d01ff30]
at org.exist.storage.NativeBroker.openCollection(NativeBroker.java:482)
- waiting to lock <0x9178d000> (a org.exist.collections.CollectionCache)
at org.exist.storage.NativeBroker.getCollection(NativeBroker.java:451)
at org.exist.xmldb.LocalCollection.getCollection(LocalCollection.java:174)
at org.exist.xmldb.LocalCollection.<init>(LocalCollection.java:130)
at org.exist.xmldb.LocalCollection.<init>(LocalCollection.java:104)
at org.exist.xmldb.DatabaseImpl.getLocalCollection(DatabaseImpl.java:230)
at org.exist.xmldb.DatabaseImpl.getCollection(DatabaseImpl.java:161)
at org.xmldb.api.DatabaseManager.getCollection(Unknown Source)
at org.xmldb.api.DatabaseManager.getCollection(Unknown Source)
at org.exist.xmldb.test.concurrent.XQueryAction.execute(XQueryAction.java:57)
at org.exist.xmldb.test.concurrent.ConcurrentTestBase$Runner.run(ConcurrentTestBase.java:169)

"Thread-28" prio=1 tid=0x08324368 nid=0x5c76 waiting for monitor entry [0x8d0a0000..0x8d0a0fb0]
at org.exist.storage.NativeBroker.openCollection(NativeBroker.java:482)
- waiting to lock <0x9178d000> (a org.exist.collections.CollectionCache)
at org.exist.storage.NativeBroker.getCollection(NativeBroker.java:451)
at org.exist.xmldb.LocalCollection.getCollection(LocalCollection.java:174)
at org.exist.xmldb.LocalResourceSet.getResource(LocalResourceSet.java:157)
at org.exist.xmldb.test.concurrent.XQueryAction.execute(XQueryAction.java:71)
at org.exist.xmldb.test.concurrent.ConcurrentTestBase$Runner.run(ConcurrentTestBase.java:169)

"Thread-27" prio=1 tid=0x08326fe0 nid=0x5c75 in Object.wait() [0x8d833000..0x8d833e30]
at java.lang.Object.wait(Native Method)
- waiting on <0x91803f30> (a org.exist.util.ReentrantReadWriteLock)
at org.exist.util.ReentrantReadWriteLock.acquire(ReentrantReadWriteLock.java:62)
- locked <0x91803f30> (a org.exist.util.ReentrantReadWriteLock)
at org.exist.storage.NativeBroker.openCollection(NativeBroker.java:522)
- locked <0x9178d000> (a org.exist.collections.CollectionCache)
at org.exist.storage.NativeBroker.openCollection(NativeBroker.java:459)
at org.exist.xmldb.LocalCollection.getResource(LocalCollection.java:336)
at org.exist.xmldb.test.concurrent.RetrieveResourceAction.execute(RetrieveResourceAction.java:43)
at org.exist.xmldb.test.concurrent.ConcurrentTestBase$Runner.run(ConcurrentTestBase.java:169)

"Thread-26" prio=1 tid=0x08322008 nid=0x5c74 in Object.wait() [0x8d730000..0x8d730eb0]
at java.lang.Object.wait(Native Method)
- waiting on <0x917f5110> (a org.exist.xmldb.test.concurrent.ConcurrentTestBase$Runner)
at java.lang.Object.wait(Object.java:474)
at org.exist.util.MultiReadReentrantLock.writeLock(MultiReadReentrantLock.java:161)
- locked <0x917f5110> (a org.exist.xmldb.test.concurrent.ConcurrentTestBase$Runner)
at org.exist.util.MultiReadReentrantLock.acquire(MultiReadReentrantLock.java:89)
at org.exist.collections.Collection.checkPermissions(Collection.java:1273)
at org.exist.collections.Collection.determineTreeStructure(Collection.java:1115)
at org.exist.collections.Collection.validate(Collection.java:863)
at org.exist.xmldb.LocalCollection.storeXMLResource(LocalCollection.java:601)
at org.exist.xmldb.LocalCollection.storeResource(LocalCollection.java:538)
at org.exist.xmldb.LocalCollection.storeResource(LocalCollection.java:530)
at org.exist.xmldb.test.concurrent.DBUtils.addXMLResource(DBUtils.java:122)
at org.exist.xmldb.test.concurrent.ReplaceResourceAction.execute(ReplaceResourceAction.java:79)
at org.exist.xmldb.test.concurrent.ConcurrentTestBase$Runner.run(ConcurrentTestBase.java:169)

"Thread-25" prio=1 tid=0x08437658 nid=0x5c6b in Object.wait() [0x8d7b1000..0x8d7b2130]
at java.lang.Object.wait(Native Method)
- waiting on <0x9178eec8> (a org.exist.storage.sync.SyncDaemon)
at org.exist.storage.sync.SyncDaemon.nextTask(SyncDaemon.java:345)
- locked <0x9178eec8> (a org.exist.storage.sync.SyncDaemon)
at org.exist.storage.sync.SyncDaemon$RunLoop.run(SyncDaemon.java:380)
at java.lang.Thread.run(Thread.java:595)

"Low Memory Detector" daemon prio=1 tid=0x080dd378 nid=0x5a54 runnable [0x00000000..0x00000000]

"CompilerThread0" daemon prio=1 tid=0x080dbe28 nid=0x5a53 waiting on condition [0x00000000..0x8ea74a98]

"Signal Dispatcher" daemon prio=1 tid=0x080daf58 nid=0x5a52 waiting on condition [0x00000000..0x00000000]

"Finalizer" daemon prio=1 tid=0x080cfbe0 nid=0x5a51 in Object.wait() [0x8eba9000..0x8eba9fb0]
at java.lang.Object.wait(Native Method)
- waiting on <0x915a6a48> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0x915a6a48> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=1 tid=0x080ceee0 nid=0x5a50 in Object.wait() [0x8ec2a000..0x8ec2ae30]
at java.lang.Object.wait(Native Method)
- waiting on <0x915a6ac8> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:474)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0x915a6ac8> (a java.lang.ref.Reference$Lock)


"main" prio=1 tid=0x0805bdf8 nid=0x5a4e in Object.wait() [0xbfffb000..0xbfffc998]
at java.lang.Object.wait(Native Method)
- waiting on <0x917f5110> (a org.exist.xmldb.test.concurrent.ConcurrentTestBase$Runner)
at java.lang.Thread.join(Thread.java:1095)
- locked <0x917f5110> (a org.exist.xmldb.test.concurrent.ConcurrentTestBase$Runner)
at java.lang.Thread.join(Thread.java:1148)
at org.exist.xmldb.test.concurrent.ConcurrentTestBase.testConcurrent(ConcurrentTestBase.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at junit.framework.TestCase.runTest(TestCase.java:154)
at junit.framework.TestCase.runBare(TestCase.java:127)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:326)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.executeInVM(JUnitTask.java:990)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.execute(JUnitTask.java:662)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.execute(JUnitTask.java:636)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:302)
at org.apache.tools.ant.Task.perform(Task.java:401)
at org.apache.tools.ant.Target.execute(Target.java:338)
at org.apache.tools.ant.Target.performTasks(Target.java:365)
at org.apache.tools.ant.Project.executeTarget(Project.java:1237)
at org.apache.tools.ant.Project.executeTargets(Project.java:1094)
at org.apache.tools.ant.Main.runBuild(Main.java:669)
at org.apache.tools.ant.Main.startAnt(Main.java:220)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:215)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:90)

"VM Thread" prio=1 tid=0x080cc2f8 nid=0x5a4f runnable

"VM Periodic Task Thread" prio=1 tid=0x080de808 nid=0x5a55 waiting on condition

2005-07-20

merge [ options ] file1 file2 file3
DESCRIPTION
       merge  incorporates  all  changes  that  lead  from file2 to file3 into
       file1.  The result ordinarily goes into file1.

        -p     Send results to standard output instead of overwriting file1.

So I did :
cd /usr/src/usr2/eXist-wolf-transac/
cp NativeBroker.java NativeBroker-wolf.java
echo ':ext:jmvanel@cvs.sourceforge.net:/cvsroot/exist' > CVS/Root
cvs update -r 1.154 NativeBroker.java

merge -p $exist/src/org/exist/storage/NativeBroker.java \
NativeBroker.java NativeBroker-wolf.java \
> NativeBroker-merge.java

merge: warning: conflicts during merge


merge -p NativeBroker-wolf.java \
NativeBroker.java \
$exist/src/org/exist/storage/NativeBroker.java \
> NativeBroker-merge2.java


http://xml.sys-con.com/read/104980.htm Indexer.storeElement(): why pushing an empty XMLString The pushed XMLString is used to collect character data while parsing the element content.

Contents