eXist journal - J.M.
Vanel
howto test eXist - eXist-1.0/test/report/html/index.html
http://drone.exist-db.org/log/existbot/existdb
: log of eXist chat
http://www.zeesource.net/maps/map.do?group=335
: geographics map of eXist developers
http://wiki.exist-db.org/
IRC channel #existdb - [cvs] / exist
/ eXist-1.0
/ src
/ org
/ exist
http://news.gmane.org/gmane.text.xml.exist
- http://news.gmane.org/gmane.text.xml.exist.cvs
Contents
2006-03-20
Error in the request logger
[giulio] jean marc
remember that there is a problem with xmlrpc and "big" documentsjmvanelYes
, I 'll add that to the wiki page .No time to do more
today and tomorrow .I propose this :
Potential problems
-
there is a problem with XML-RPC when uploading "big" documents ( around
1Mb or even less ), e.g. using the GUI : the log seems to be incomplete
[giulio]no jean marc
also with very little documents 10-15 kb
jmvanel
giulio , so the problem is when uploading any document with XML-RPC
using one of the methods in
http://exist.sourceforge.net/devguide.html#N106BC
?[giulio]jean
marc the pronblem is not strictly to xmlrpc but how the message tcp is
send when the document is not complete with a single tcp messagein
general you can have problem with method exposebut how
yesterday i write-->|wolf77 (n=opera@L28bf.l.pppool.de)
has joined #existdb[giulio]the problem is
that ceck the lenght of the message is not good
[giulio] |
is
like you have a xml document with start node and no end node |
|
jmvanel |
Giulio,
when I look at the original RpcServlet , it just does
request.getInputStream()
and inputs that into xmlrpc.execute() |
|
[giulio] |
yes
and in this is correct |
|
jmvanel |
Sothis
is why I think that request.getInputStream() is able to
gather physical message into a single logical message |
|
[giulio] |
yes
ok |
|
the
wrapper accept one message and pass it to xmlrpc server |
|
but
without know if this is complete or no |
|
or
also can be possible that the wrapper close the connection so nothing
also arrive |
|
jmvanel |
It
may be that
is.read(contentBody);
in HttpServletRequestWrapper
doesn't real all and should be in a loop . |
|
_wolf77_ |
chongma:
you need to cast the object |
|
jmvanel:
yes, check for example RESTServer.getRequestContent |
|
jmvanel |
OK.
Giulio , yes I think that's the problem. Could you try to replace
is.read(contentBody);
line 294 in HttpServletRequestWrapper
with
int bytes = 0;
int offset = 0;
int max = 4096;
while (( bytes = is.read( contentBody, offset, max )) != -1) {
offset += bytes;
} |
2006-03-19
Extention of HTTP+requests+logger to other protocols
http://wiki.exist-db.org/space/HTTP+requests+logger
Test with WebDav
cadaver
dav:!> open http://localhost:8080/exist/webdav/db
Authentication required for exist on server `localhost':
Username: guest
Password:
dav:/exist/webdav/db/> ls
Listing collection `/exist/webdav/db/': succeeded.
Coll: XQTS 0 Mar 18 17:01
Coll: complex 0 Mar 17 20:27
Coll: test 0 Mar 17 20:28
Coll: testCollectionExists 0 Mar 18 19:00
Coll: �t� 0 Mar 18 18:10
dav:/exist/webdav/db/> put build.xmL
Uploading build.xmL to `/exist/webdav/db/build.xmL': Could not open file: No such file or directory
dav:/exist/webdav/db/> put build.xml
Uploading build.xml to `/exist/webdav/db/build.xml':
Progress: [=============================>] 100.0% of 56204 bytes succeeded.
2006-03-15
Tests with gcj
library + eclipse
Only 5 errors left when linking eXist to the gcj libraries !
- 2 are in the GUI
- 2 are in a test
- the last concerns time values only
Severity Path
Resource Description
Location Creation Time Id
Error
eXist-1.0/src/org/exist/client
DocumentView.java The constructor
OutputStreamWriter(FileOutputStream, Charset) is
undefined line 435 mars 15, 2006
7:23:20 3192
Error
eXist-1.0/src/org/exist/client
InteractiveClient.java The constructor
OutputStreamWriter(FileOutputStream, Charset) is
undefined line 2370 mars 15, 2006
7:23:19 3180
Error
eXist-1.0/src/org/exist/http/test
RESTServiceTest.java BASE64Encoder cannot be resolved
to a type line 87 mars 15, 2006
7:22:38 2939
Error
eXist-1.0/src/org/exist/http/test
RESTServiceTest.java The import sun cannot be
resolved line 44 mars 15, 2006
7:22:38 2938
Error
eXist-1.0/src/org/exist/xquery/value
TimeUtils.java The method newDurationDayTime(boolean,
BigInteger, BigInteger, BigInteger, BigDecimal) in the type
DatatypeFactory is not applicable for the arguments (boolean,
BigInteger, BigInteger, BigInteger, BigInteger) line
95 mars 15, 2006 7:19:54 1639
So a test is possible with the gcj compiler too . Fedora Core 4 offers
a convenient environment, the "native eclipse" .
But first I try with eXist compiled by normal eclipse, plus the GNU jre
( gcj libraries). There is a problem with Cocoon :
java.lang.reflect.InvocationTargetException
at java.lang.reflect.Method.invoke(java.lang.Object, java.lang.Object[]) (/usr/lib/libgcj.so.6.0.0)
at org.exist.start.Main.invokeMain(java.lang.ClassLoader, java.lang.String, java.lang.String[]) (Unknown Source)
at org.exist.start.Main.run(java.lang.String[]) (Unknown Source)
at org.exist.start.Main.main(java.lang.String[]) (Unknown Source)
at gnu.java.lang.MainThread.call_main() (/usr/lib/libgcj.so.6.0.0)
at gnu.java.lang.MainThread.run() (/usr/lib/libgcj.so.6.0.0)
Caused by: java.lang.NoClassDefFoundError: while resolving class: org.apache.cocoon.components.flow.javascript.fom.FOM_JavaScriptInterpreter
at java.lang.VMClassLoader.transformException(java.lang.Class, java.lang.Throwable) (/usr/lib/libgcj.so.6.0.0)
at java.lang.VMClassLoader.resolveClass(java.lang.Class) (/usr/lib/libgcj.so.6.0.0)
at java.lang.Class.initializeClass() (/usr/lib/libgcj.so.6.0.0)
at java.lang.Class.isAssignableFrom(java.lang.Class) (/usr/lib/libgcj.so.6.0.0)
at org.apache.avalon.excalibur.component.ComponentHandler.getComponentHandler(java.lang.String, java.lang.Class, org.apache.avalon.framework.configuration.Configuration, org.apache.avalon.framework.component.ComponentManager, org.apache.avalon.framework.context.Context, org.apache.avalon.excalibur.component.RoleManager, org.apache.avalon.excalibur.component.LogkitLoggerManager, org.apache.excalibur.instrument.InstrumentManager, java.lang.String) (Unknown Source)
This is caused by a class not found that I can't find nowhere ( in JRE
or in eXist ) with :
for f in **/*.jar; do echo $f; unzip -l $f | \
grep org.mozilla.javascript.tools.debugger.Main; done
So let's suppress Cocoon and test the eXist servet with :
http://localhost:8080/exist/xquery/guess.xql
To suppress Cocoon I comment out in web.xml :
<servlet-mapping>
<servlet-name>Cocoon</servlet-name>
<url-pattern>/</url-pattern>
</servlet-mapping>
In fact I need to suppress the <servlet> tag too.
I had one more problem with JAF, that I solved by adding this to
the eclipse project libraries :
/usr/share/java/jaf.jar
Youpeeeeeeeeeeeeeeeeee !
Exist ( i.e. guess.xql ) runs with the gcj libraies !
However, trying the sandbox
http://localhost:8080/exist/sandbox/sandbox.xql
with /* (and nothing in the DB) gives problems:
org.xml.sax.SAXParseException@53947f8
at org.exist.Indexer.fatalError(org.xml.sax.SAXParseException) (Unknown Source)
at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(java.lang.String, java.lang.String, org.apache.xerces.xni.parser.XMLParseException) (Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(org.apache.xerces.xni.XMLLocator, java.lang.String, java.lang.String, java.lang.Object[], short) (Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(java.lang.String, java.lang.String, java.lang.Object[], short) (Unknown Source)
at org.apache.xerces.impl.XMLScanner.reportFatalError(java.lang.String, java.lang.Object[]) (Unknown Source)
at org.apache.xerces.impl.XMLDocumentScannerImpl$PrologDispatcher.dispatch(boolean) (Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(boolean) (Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(boolean) (Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(org.apache.xerces.xni.parser.XMLInputSource) (Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(org.apache.xerces.xni.parser.XMLInputSource) (Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(org.xml.sax.InputSource) (Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(org.xml.sax.InputSource) (Unknown Source)
at org.exist.collections.Collection$4.run(org.exist.collections.IndexInfo) (Unknown Source)
at org.exist.collections.Collection.validateXMLResourceInternal(org.exist.storage.txn.Txn, org.exist.storage.DBBroker, java.lang.String, org.exist.collections.Collection$ValidateBlock) (Unknown Source)
at org.exist.collections.Collection.validateXMLResource(org.exist.storage.txn.Txn, org.exist.storage.DBBroker, java.lang.String, org.xml.sax.InputSource) (Unknown Source)
at org.exist.xmldb.LocalCollection.storeXMLResource(org.exist.xmldb.LocalXMLResource) (Unknown Source)
at org.exist.xmldb.LocalCollection.storeResource(org.xmldb.api.base.Resource, java.util.Date, java.util.Date) (Unknown Source)
at org.exist.xmldb.LocalCollection.storeResource(org.xmldb.api.base.Resource) (Unknown Source)
at org.exist.xquery.functions.xmldb.XMLDBLoadFromPattern.evalWithCollection(org.xmldb.api.base.Collection, org.exist.xquery.value.Sequence[], org.exist.xquery.value.Sequence) (Unknown Source)
at org.exist.xquery.functions.xmldb.XMLDBAbstractCollectionManipulator.eval(org.exist.xquery.value.Sequence[], org.exist.xquery.value.Sequence) (Unknown Source)
at org.exist.xquery.BasicFunction.eval(org.exist.xquery.value.Sequence, org.exist.xquery.value.Item) (Unknown Source)
at org.exist.xquery.InternalFunctionCall.eval(org.exist.xquery.value.Sequence, org.exist.xquery.value.Item) (Unknown Source)
at org.exist.xquery.PathExpr.eval(org.exist.xquery.value.Sequence, org.exist.xquery.value.Item) (Unknown Source)
at org.exist.xquery.DynamicCardinalityCheck.eval(org.exist.xquery.value.Sequence, org.exist.xquery.value.Item) (Unknown Source)
at org.exist.xquery.AbstractExpression.eval(org.exist.xquery.value.Sequence) (Unknown Source)
As you can see, the problem is also that the GNU jre ( gcj libraries)
has no debugging informations .
What does eXist try to store ? Probably the sandbox/xml-highlight.xsl
sylesheet .
This problem with the xerces parser seems to be well-know in the
eXist team ?
The version I have in /usr/share/java is :
xerces-j2-2.6.2-4jpp_5fc
And eXist has Xerces 2.7.1
TO BE CONTINUED .
Extending Adam Retter
(delirium)'s request logger
src/org/exist/http
Added Files:
Descriptor.java
Log Message:
Added the facility for a descriptor.xml file to detail path mappings for the REST interface.
Usefull for default document like functionality. Enables mappings of folder->file, folder->folder, file->folder, file->file
The replayer is in
tools/requestlog
2006-03-06
I'm debugging XQueryPool with XML-RPC , the GUI client, and always the
same
query .
I have the inmpression that the compiled query is never reused .
Also I don't understand the reason to have a Stack inside the values[]
array
.
The test line 138 is never true :
Back from a long discussion with my eXist client ...
- He 'd like to have joins optimized :-)
- he saw that queries seem twice as quick on Tomcat than on the
bundled Jetty server
- he wants to refine queries; is the first query kept in cache and
reused ?
- is the cluster used in production ?
- is order by using the index when present ?
[23:10:24] <wolf77> the last two questions are easy to answer:
[23:11:12] <wolf77> order by does not use an index. it would
require a
specific type of index that allows fast access to values by node ids.
[23:11:30] <wolf77> but I would be interested to see such an
index
added.
[23:12:12] <wolf77> the cluster was developed by a commercial
company for a production system.
[23:12:20] <wolf77> I thus assume it is used in production.
[23:12:41] <[giulio]_x> yes wolf about the cluster in production
[23:12:56] <[giulio]_x> but I suspect the development is for a
specific
contenxt
[23:13:02] <[giulio]_x> like small documents
[23:13:44] <[giulio]_x> I write this because in my test i see
that if
the message send is not so big
[23:13:59] <[giulio]_x> the cluster work otherwise after a little
there
a re problem of syncro
[23:14:12] <wolf77> yes, that's very much possible. I also think
they
did not address queries to the database because they had a hardware
proxy
doing this.
[23:14:20] <[giulio]_x> yes
[23:14:45] <[giulio]_x> If I remember the developper write a day
that
have an hardware for do this
[23:17:48] <_jmvanel_> Thank you all.
[23:17:48] <_jmvanel_> And what about refining queries ?
[23:17:48] <_jmvanel_> First query :
[23:17:48] <_jmvanel_> /tag[constraint1]
[23:17:48] <_jmvanel_> Second query :
[23:17:48] <_jmvanel_> /tag[constraint1][constraint2]
[23:18:01] <_jmvanel_> is the first query kept in cache and
reused ?
[23:18:17] <wolf77> Clear answer: no
Made some trials with AspectJ ( see my general blog
).
2006-02-22
Profiling with
eclipse TPTP
Using the latest sub-release ( not daily build). One needs to download
the
eclipse plugin, plus the special OS-specific server . Then you have to
go in
"profiling" tab in window Run/profile , and do "Add" to add the
analysis that
you want . I chose:
- basic memory analysis
- Execution time analysis
- Method code coverage
I ran this query on the XQTS definition file :
declare namespace cat= ... ;
//cat:test-case [ @name="Literals002" ]
Here it the time analysis result
. The
base time is the most interesting . Note that it took a very time, I
was
obliged to let it run all night.
Here are memory analysis:
Alas the export as CSV or HTML files crashed !
[09:21:37] <wolf77> I guess the journal and sync timings are
explained
by the query taking all night. eXist has probably done quite a lot of
syncs
over night ;-)
[09:21:53] <_jmvanel_> Yes, sure.
[09:21:57] <wolf77> Compare http://exist-db.org/files/profile.html
[09:21:58] <wolf77> Same query.
[09:22:41] <wolf77> As expected, most of the time is spent in
NodeProxy.atomize(). No surprise: there's no index defined.
[09:22:51] <ljo> Well, just asking since you said it should not
be used
at all.
[09:23:14] <wolf77> Yeah, I was first wondering as well.
[09:36:26] <_jmvanel_> Comparing the results of YourKit , and
mine ;-)
, that is TPTP, I see that for
[09:36:26] <_jmvanel_>
GeneralComparison.quickNodeSetCompare(Sequence)
[09:36:26] <_jmvanel_> The base time given TPTP is very low,
whereas
Yourkit shows 19% ; does this functions really just do
[09:36:26] <_jmvanel_> some delegation ?
2006-02-19
Versailles 2006 eXist
meeting
My presentation (
OpenOffice.org
2 format ), My presentation
in
PDF
Dannes Wessels' (DiZzZz)
presentation
in PPT, DiZzZz)
presentation in
PDF
Wolfgang's two presentations : eXist architecture + history (in
english) ;
eXist algorithms (in german) .........
Pictures taken by DiZzZz http://dilemma.ow.nl/versailles2006/
My notes (saturday 18)
..........................
My notes (sunday 19)
PSVI - easier with JAXP
reference Schema cached - problem with non-existing schema makes eXist
slow
distrib. without Cocoon ; doc. in the database;
new Ant targets: embedded eXist, plain server
SleepyCat better for data-centric apps
Wolf:
store 100000 files
potential for local optimisations in a lot of places
commit only bugs on the HEAD branch
test
creating small data by program, and storing it 100000 times
Perig:
use XQTS data structure, submit our tests to XQTS
need low-level tests
Wolf:
transaction tests also tests BFile
transform tests into stress tests by adding parametrizable loops
tests regressions in XQuery: store in the DB the reference profiles,
and
compare the to the actual profile
JUnit 4 : overall setUp and tearDown
TestNJ
using Java 5 features and generating Java 1.4 byte code ?
making conf.xml dynamic
Spring use: configuration ; class instantiation
pool feature?
NodeNumbering interferface for DLN
key: collection id, real key --> node id
basic design decision in eXist : node set versus traversal
Jaxen?
eXist: context + input node set --> result
reused from Saxon: serialization, functions implementation
Prague: 5 hours 9-2 o'clock
Wiki page;
development team : one hour, roadmap
orbeon 3 + javascript
Xforms implementation in XQuery : not full, needs documentation
XQuery inside Xforms
MVC pattern : controller = XQuery
Perig: explore XUL runner (Mozilla)
september : Monet conference, in-memory DB, meta-language internally,
XML : 2
id's internally
documentation : extended XQ features, modules TODO,
Webdav DONE
TODO conf.xml ; enhance indexing, using XQdoc
sf compile farm ; continous build system
XQTS is not quite finished, but it was a team effort, with Wolfgang and
me
writing the main script,
and Perig and Dannes writing a new function to define a context
by
program.
2006-02-17
Preparing eXist meeting ...
java -cp tools/lib/jdepend-2.9.jar jdepend/swingui/JDepend build/classes
Added this at begining of /etc/ant.conf
# jmv
return
2006-02-16
do you know what schemes wolf is looking at?
[10:00:55] <_jmvanel_> http://dbs.uni-leipzig.de/en/projekte/XML/Boehme_DLN_DIWeb_CR.pdf
DLN = dynamic level numbering : Hourrah, Wolfgang commited it
!
howto work with the
"new
numbering" CVS branch
I went to "configure tags" when in CVS repository exploring,
I navigated to NativeBroker, and I added "DLN" to the list of
remembered
tags.
Then I created a new eclipse project from the usual eXist-1.0 first
level
directory, but with the branch tag DLN .
Note that this CVS branch (and hence the project) contains only the
directory
src/ , no lib/ , no webapp/ , etc .
So I made the new eclipse project ( I called it eXist-1.0_dln)
dependant on
the normal eclipse project "eXist-1.0" . But that 's not enough to get
the
libs from the normal eclipse project "eXist-1.0" . You must also add
all
third party libs from the normal eclipse project to the exported
libraries in
tab "Java build path / order and export" .
Then all should compile .
Note that this way Wolfgang is not obliged to put all sources in the
branch
DLN , as he done now . He could put on the new branch just the modified
files
.
Then to execute with the GUI client in embedded mode , I must copy or
link
the missing files :
# linking or creating necessary files from the CVS HEAD:
mkdir --parents webapp/WEB-INF/data/
ln -s $exist/conf.xml .
ln -s $exist/client.properties .
ln -s $exist/webapp/WEB-INF/catalog.xml .
ln -s $exist/webapp/WEB-INF/log4j.xml .
Variable $exist represents the location of the normal eclipse project .
That's all .
Using this to kompare the old and new code :
kompare src ../eXist-1.0-sync/src &
My fisrt stack with the new numbering scheme :
Reading a comment when storing hamlet.xml :
java.lang.NullPointerException
at
org.exist.dom.CommentImpl.serialize(CommentImpl.java:55)
at
org.exist.storage.NativeBroker$15.start(NativeBroker.java:2247)
at
org.exist.storage.dom.DOMTransaction.run(DOMTransaction.java:56)
at
org.exist.storage.NativeBroker.storeNode(NativeBroker.java:2263)
at
org.exist.storage.DBBroker.storeNode(DBBroker.java:483)
at org.exist.Indexer.comment(Indexer.java:219)
Allready solved by added 1 line in Indexer !
[16:52:35] <wolf77> Jean-Marc: queries don't work with the new
numbering scheme yet.
[16:52:52] <wolf77> all the NodeSet methods need to be changed
first.
[16:53:05] <wolf77> all you can do now is to store/retrieve
documents.
[17:00:21] <_delirium> wolf77: hey wolf, did you have a chance to
look
at the HTTP 500 error problems in that email I sent you?
[17:00:38] <wolf77> also, parts of the indexes do already store
the new
node ids, other parts still use the old one. you can't expect that to
work
;-)
[17:02:25] <wolf77> _jmvanel_: I calculate at least another 8
weeks
until the test suite will run with the new scheme.
Adding to the
"Incompatible
changes", in the change log.
[19:15:27] <_jmvanel_> Want to add "no more implicitly global
variables"
[19:15:27] <_jmvanel_> to "Incompatible changes", in the change
log.
[19:15:27] <_jmvanel_> Was that on the 24th snapshot too ?
before eXist would let you write $a[1 to 10], you now need to write
$a[(1 to
10)]
2006-02-13
Commited feature
"XQuery
trigger with update" (not quite finished)
|
|
|
09:30:29 |
_jmvanel_ |
Wolf, debugging in
Modification.selectAndLock() , lines: |
09:30:30 |
_jmvanel_ |
NodeList nl = select(docs); |
09:30:30 |
_jmvanel_ |
lockedDocuments = ((NodeSet)
nl).getDocumentSet(); |
09:30:30 |
_jmvanel_ |
nl has just one node, but |
09:30:30 |
_jmvanel_ |
getDocumentSet() returns its cachedDocuments
, that includes all collection. |
09:30:30 |
_jmvanel_ |
I think it's not normal. |
09:32:38 |
wolf77 |
Yes, that's rather strange. |
09:36:32 |
_jmvanel_ |
Can be seen in
XQueryTriggerTest.testUpdateDocument() , but with the new |
09:36:33 |
_jmvanel_ |
update Trigger on my site . |
My breakpoints :
Modification [line: 101] - nullDocumentTrigger
Modification [line: 214] - selectAndLock()
Modification [line: 283] - getTrigger(DocumentImpl)
TriggerStatePerThread [line: 125] - setTriggerRunningState(int,
DocumentTrigger, DocumentImpl)
2006-02-05
[20:39:27] <_jmvanel_> About to add this field in XQueryTrigger
[20:39:27] <_jmvanel_> private static ThreadLocal triggerRunning
= new
ThreadLocal() {
protected synchronized Object initialValue() {
return Boolean.FALSE;
}
[20:39:27] <_jmvanel_>
[20:39:27] <_jmvanel_> and this call at the beginning of
XQueryTrigger.prepare()
[20:39:27] <_jmvanel_> triggerRunning.set(Boolean.TRUE);
[20:39:27] <_jmvanel_> and the converse at the end of
finish()
2006-01-28
appendChildren(Txn transaction, NodeList nodes, int child)
[[13:39:53] <Perig> Well... trying to adress this XUpdate bug
[13:40:21] <Perig> Interesting one whic will probably lead us in
the
very core of the code :-)
[13:41:13] <_jmvanel_> I was also looking at
http://skaringa.sourceforge.net/doc.html#i21219507
[13:41:37] <[giulio]_> perig the "yes" about omit xquery
diagnostics is
for me?
[13:41:45] <_jmvanel_> How could I help Perig ?
[13:42:04] <Perig> giulio : what are you talking about ?
[13:42:17] <Perig> _jmvanel_ in writing a simple test case :-)
[14:47:45] <_jmvanel_> Where in the W3C docs do they say that
adjacent
nodes are necessarily glued as one ?
[14:49:47] <[giulio]_> sorry Perig I'am in wrong
[14:50:22] <Perig> Looking for the spec to make everything clear
[14:52:47] <_jmvanel_> I remember that the spec is tolerant about
that
; the glueing may occur or not .
[14:53:06] <_jmvanel_> But not 100% shure.
[14:53:07] <Perig> http://www.w3.org/TR/xpath#section-Text-Nodes
[14:53:17] <Perig> Old Xpath specs.
[14:53:43] <Perig> Trying to get the up to date data model
[14:54:28] <Perig> OK. Clear :
[14:54:30] <Perig> http://www.w3.org/TR/xpath-datamodel/#TextNode
[14:55:20] <Perig> So, the user (and myself) is right IMHO. Do
you
agree ?
...
[15:02:05] <_jmvanel_> Perig of course you're right about the
meaning
of the spec regarding adjacent text nodes: they are forbiden.
[15:02:11] <Perig> OK. But I'm trying to find a definitive
solution
15:03:14] <_jmvanel_> I can write a JUnit test method; do you
have a
preference for an existing test class ?
[15:03:25] <Perig> In clear, eXist sould *never ever* allow
sibling
text nodes.
[15:03:58] <_jmvanel_> Might be difficult but yes that's the goal.
[15:04:33] <Perig> Major deficiency IMHO. Node IDs are precious
:-)
[15:05:18] <Perig> ElementImpl seems to be a good candidate though
[15:07:07] <Perig> (trigger a ReplaceChild rather than an
AppendChild)
[15:07:19] <_jmvanel_> naively I would say that glueing nodes is
easier
than splitting, because we don't consumme precious ID intervals.
[15:07:36] <_jmvanel_> Which line ?
[15:08:02] <Perig> ElementImpl, line 449
[15:08:34] <Perig> but 438 and 446 may be concerned as well.
[15:09:30] <Perig> coalescing nodes it not that easy (even if
there are
more difficult things) because we must test that we have text nodes...
[15:09:52] <Perig> whereas eXist tries to avoid such tests
[15:11:29] <_jmvanel_> I see. Have you looked at TextImpl ?
[15:12:00] <Perig> Not yet. Still wondering who should have
control
:-)
[15:12:23] <_jmvanel_> BTW, don't you think we should once and
for all
apply eclipse format on whole eXist and commit that ?
[15:12:39] <_jmvanel_> Democratically , of course ...
[15:12:47] <Perig> In short : should the parent check that it
doesn not
have 2 consecutive children ?
[15:13:06] <Perig> Or... the node that it does not have a sibling
?
[15:13:13] <_jmvanel_> These lines with 119 characters are
bothering
.
[15:13:43] <Perig> Jean Marc : I'd support such an initiative.
[15:13:51] <_jmvanel_> The latter avoids the type test ...
[15:13:59] <Perig> But I'am rather used to "Apache" coding rules.
[15:15:36] <_jmvanel_> OK. interesting , but let's settle that
after
...
[15:16:14] <Perig> No problem : I want to take some time with
this text
node problem.
[15:16:39] <_jmvanel_> I'll have a look at TextImpl, but I must
tell
that we have a similat class hierarchy for memtree, alas.
[15:17:09] <Perig> I'm also used to these :
http://www.gnu.org/software/classpath/docs/hacking.html#SEC6
[15:17:41] <Perig> Sure : we clearly have 2 parallel hierarchies
[15:18:10] <Perig> Nice place for design pattern governed
refactorings
;-)
[15:18:41] <_jmvanel_> I remember that Wolf wrote that he got
inspiration from Saxon.
[15:19:10] <_jmvanel_> But why not reuse Saxon for the in-memory
trees?
[15:19:25] <Perig> he, he ;-)
[15:19:46] <_jmvanel_> Some discussions lately that I missed ?
[15:20:38] <Perig> No strategic ones.
[15:21:03] <Perig> Just that wolf has a milestone about
developing a
more robust numbering scheme
[15:21:28] <_jmvanel_> Anyway, bebore being intelligent, let me
write
my JUnit ; so in case of need, I'll be able to debug ...
[15:22:40] <_jmvanel_> About the numbering scheme , I saw
and article
about the ReiserFS File System in Linux Magazine (en
Français).
[15:22:53] <_jmvanel_> Maybe some ideas to pick ...
[15:26:35] <Perig> Wolfgang told me that he was relyng on
variable
lenghth keys.
[15:27:09] <Perig> What I'd like is (yet another) pluggable thing.
[15:30:05] <_jmvanel_> :-)
[15:30:10] _delirium [n=adam@cpc1-derb1-6-0-cust26.leic.cable.ntl.com]
a
quitté IRC : "using sirc version 2.211+KSIRC/1.3.12"
[15:30:51] <Perig> Gev made a suggestion that at first glance
looked
crazy : store all the possible relationships between nodes.
[15:31:20] <Perig> However... storing them in a cache beginds to
make
sense
[15:32:55] <Perig> A plugabble thing would also ease what has
been
requested recently on the mailing-list : traverse XIncludes.
[15:34:16] <Perig> That is, in a given execution scope, node X
from
document A is to be viewed as the parent of node Y in docment B
[15:35:04] <_jmvanel_> :-) flexibility points sure guide the
design
.
[15:35:26] <Perig> I hope they will...
[15:36:50] <_jmvanel_> Another idea : uncouple the storage and
the
indexing, so that one can keep the XML, e.g., in a plain file system,
with
indices that point to interesting places in this storage.
[15:37:23] <Perig> Oh yes :-)
[15:38:03] <Perig> A design close to SDX's BTW
[15:38:15] <_jmvanel_> haha
[15:38:44] <_jmvanel_> So we need abstractions of :
[15:38:51] <_jmvanel_> - storage
[15:38:56] <_jmvanel_> - indexer
[15:39:29] <_jmvanel_> - editable storage
[15:40:14] <Perig> indexerSSSS
[15:40:54] <Perig> an easy abstraction would be to consider
stored item
as a ache entry
[15:41:05] <Perig> That how Gev's application works :-)
[15:46:11] <Perig> BTW : anybody remembers that "binary" XML
storage
(from China ?) that store the hierarchy in the file itself ?
[15:48:01] <_jmvanel_> No. What is " a ache entry" ?
[15:49:15] <Perig> cache
[15:50:21] <Perig> In clear : use EH cache in place of the native
BFiles :-)
[15:52:47] <[giulio]_> so if the file is not changed there are no
serilization?
[15:53:29] <_jmvanel_> I know nothing yet about EH cache, except
that
it's widely used.
[15:54:09] <Perig> Yes : now used by Cocoon
[15:55:16] <_jmvanel_> The least we can do is having a clean
separation
between "low-level" storage and XML storage, which brings as back to
the
indexing scheme ...
[15:55:34] <_jmvanel_> us back
[15:56:08] <Perig> Yes. The StoredNode/NodeProxy decoupling is a
good
step forward
[15:58:07] <_jmvanel_> yes.
[16:01:36] <Perig> But NodeProxy has still an internalAddress :-(
[16:03:01] <_jmvanel_> Alas.
[16:03:01] <_jmvanel_>
[16:03:01] <_jmvanel_> Maybe some ideas can also be gained from
Jaxen
:
[16:03:01] <_jmvanel_> http://jaxen.codehaus.org/faq.html
[16:03:01] <_jmvanel_> I just read the book about PMD; they use
Jaxen
to query by XPath the Abstract Syntax Tree.
[16:03:01] <_jmvanel_> The AST does not implement any DOM or Jax
interfaces, but iterators for
[16:03:01] <_jmvanel_> the XPath axis .
[16:24:28] <_jmvanel_> I propose to put this in XPathQueryTest
Javadoc:
[16:24:28] <_jmvanel_> tests involving only XPath expressions,
[16:24:28] <_jmvanel_> and in XQueryTest
[16:24:28] <_jmvanel_> tests involving the others constructs of
the
XQuery language
[16:24:28] <_jmvanel_> besides XPath expressions
[16:24:28] <_jmvanel_> public class XQueryTest
[16:25:56] <Perig> Some serious clean ups there ;-)
[16:28:36] <_jmvanel_> But do you agree with the classification
principle ?
[16:30:58] <Perig> Yes (although I think that all this kind of
tests
should be handled by the XQTS)
[16:32:58] <_jmvanel_> XQTS ?
[16:45:22] <_jmvanel_> Anyway, Perig, we got our bugtest in
XQueryTest.java ...
[16:51:50] Perig [n=chatzill@vil35-2-82-236-174-106.fbx.proxad.net] a
quitté
IRC : Read error: 113 (No route to host)
[16:53:48] <_jmvanel_> When we fix thzt, we can correct the
misspelling:
[16:53:48] <_jmvanel_> bugtestXUpdateWithAdvancentTextNodes() ;-)
[16:54:16] <_jmvanel_> Adjacent
[16:57:27] <_jmvanel_> No least than 6 functions to add one or
more
child in ElementImpl :-( !
[17:16:42] <_jmvanel_> Still trying to orientate in the
ElementImpl.appendXXX() jungle;
[17:16:42] <_jmvanel_> as a boussole I have the call graph.
[17:16:42] <_jmvanel_> But my laptop screen is too small, have
use my
1600x1200 screen !
[17:17:36] <_jmvanel_> And ElementImp is 1500 lines, should be
split as
least in 3 !
[17:31:42] <_jmvanel_> To see all the little white stones you've
put
... Nice!
[17:31:42] <_jmvanel_> When will there be an oportunity to review
all
that with Wolf in flesh ?
[17:32:05] <Perig> :-)
[17:33:07] <_jmvanel_> Will you be in Prague ?
2006-01-25
I saw nice refactorings in the storage/ package :
- turn Paged into a delegate in BTree , etc
- apply "state" design pattern to
org/exist/storage/btree/Paged.java (commited comment)
I tried to make a JUnit test for BTree. It 's not obvious, because of
configuration that must be in a file, and of the undocumented methods,
e.g.
addValue() .
2005-12-07
Problems with French
accents in HTTP GET
When I do this request :
http://localhost:8080/exist/xquery/test-param.xql?param=%E9t%E9
with this simple XQuery test-param.xql :
(: declare option exist:encoding "UTF-8"; "iso-8859-1"; :)
declare variable $param { request:request-parameter("param", "") };
<html>{ $param }</html>
I get this result :
<html>ᅵtᅵ</html>
When debugging, I saw in HttpRequestWrapper.decode()
that containerEncoding == formEncoding == UTF8
Here is how my string 'été'
is when it goes out of the J2EE
function HttpServletRequest.getParameterValues(key)
[17:00:14] <|jmvanel|>
(int)values[0].charAt(0)
[17:00:14] <|jmvanel|> (int) 65533
[17:00:14] <|jmvanel|>
(int)values[0].charAt(1)
[17:00:14] <|jmvanel|> (int) 116
[17:00:14] <|jmvanel|>
(int)values[0].charAt(2)
[17:00:14] <|jmvanel|> (int) 65533
Don't know if it's normal, but looks like UTF16
I came to this because my main problem comes in a "real" XQuery when I
compare some content in the DB with the string from the HTTP parameter .
When I try this with param=é everything is normal
; result is "true"
:
declare variable $param { request:request-parameter("param", "") };
<html>{
<param>{$param}</param> ,
"e accent ", "'é'" ,
"e accent eq $param",
"é" eq $param
}</html>
The page returned is UTF-8 , text/html . One can see again that the 2
é are not output right .
2005-11-25
I have corrected some bugs on my Web test GUI .
It's really useful for me .
My web test gui is a CVS module inside eXist , so, to download
it :
cvs -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/exist login
cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/exist co -P web_test_gui
There's no build.xml . I use eclipse; the eclipse
project is in CVS .
The main class is WebTestFrame. The CVSWeb is here :
http://cvs.sourceforge.net/viewcvs.py/exist/web_test_gui/src/org/exist/webtest/
The next thing I want to add are some user comments on each test URL ,
and
while doing that I will create a class TestURL, now it's just a String .
2005-11-12
Non-regression
GUI
tester for multiple URLs
I commited my my non-regression GUI tester; see more details below .
To use it ;
cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/exist co web_test_gui
Optionaly create a new eclipse project from the directory
web_test_gui
Start the class org/exist/webtest/WebTestFrame
if one leaves the "source directory" empty, no management of original
source
file will be done .
2005-11-08
Interesting thread on the list :
Re: [Exist-open] eXist vs SleepycatMichael Beddow replies longly on
this .
Non-regression
GUI
tester for multiple URLs
I did a non-regression tester for multiple URLs implemented by
XQuery
source.
I works for me ; If some of you find it useful, I can share it as a new
module in eXist CVS, or elsewhere. I used it to :
- compare results when optimiziing a bunch of XQuery scripts having
common modules
- compare results of a bunch of XQuery scripts between two eXist
versions
Screenshot :
webtest-gui.png
Download eclipse auto-sufficient project :
webtest.zip
- uptodate
version is in eXist CVS :
cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/exist co web_test_gui
What exactly can I do with it?
You can enter some relative URLs ; then the program
will save the
original XQuery source and result. After that you can modify (optimize
or
refactor) your xqueries , and push "check all URLs", and XMLUnit checks
all
results .
Jwhat do I have to add
into "Relative test URL"? |
|
|
Look the tooltip :-) |
|
|
can be terninated or not
by / , |
|
|
and the relative URL's
must begin or not by / accordingly . |
look please httt://www.rosavtotorg.ru |
|
wolf77 |
yeah, it's getting nicer
and nicer ;-) The map selection is cool. |
|
gev |
thanks |
|
jmvanel |
Yes, gev, it looks good ;
is it a 100% eXist application, no JSP, no JDBC, no LDAP ? |
|
giulio692 |
Gev very nice and fast |
|
gev |
yes it's 100% eXist
application. I only add batik library and cocoon's svg generator to
rasterize svg images |
|
gev |
also I use cocoon's
features to scale images |
2005-10-20
count ( tokenize( "", "") )
returns 1 !!!!!!!!!!!
<!-- jmv -->
<map:match pattern="*/**">
<map:mount check-reload="yes" src="{1}/sitemap.xmap"
uri-prefix="{1}" />
</map:match>
When starting GUI client in local mode :
Caused by: org.exist.EXistException:
org.exist.storage.btree.DBException:
Failed to open database file:
/mnt/usr/src/usr2/eXist-1.0/webapp/WEB-INF/data/dom.dbx. It is locked
by
another process.
at
org.exist.storage.NativeBroker.<init>(NativeBroker.java:249)
at
org.exist.storage.BrokerFactory.getInstance(BrokerFactory.java:34)
at
org.exist.storage.BrokerPool.createBroker(BrokerPool.java:855)
at
org.exist.storage.BrokerPool.initialize(BrokerPool.java:606)
at
org.exist.storage.BrokerPool.<init>(BrokerPool.java:542)
at
org.exist.storage.BrokerPool.configure(BrokerPool.java:158)
at
org.exist.xmldb.DatabaseImpl.configure(DatabaseImpl.java:128)
DOMFile(Paged).setFile(File) line: 392
DOMFile.<init>(BrokerPool, File, CacheManager) line: 180
NativeBroker.<init>(BrokerPool, Configuration) line: 215
BrokerFactory.getInstance(BrokerPool, Configuration) line: 34
FileChannel
channel = raf.getChannel();
if
(channel.tryLock() == null)
throw new DBException("Failed to open database file:
" +
file.getAbsolutePath() +
". It is
locked by another process.");
When starting GUI client :
org.xmldb.api.base.XMLDBException: org.apache.xmlrpc.XmlRpcException:
Invalid
character data corresponding to XML entity 
at
org.exist.xmldb.RemoteCollection.getResource(RemoteCollection.java:290)
at
org.exist.client.InteractiveClient.getResources(InteractiveClient.java:320)
at
org.exist.client.InteractiveClient.run(InteractiveClient.java:2139)
at
org.exist.client.InteractiveClient.main(InteractiveClient.java:222)
at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.exist.start.Main.invokeMain(Main.java:119)
at org.exist.start.Main.run(Main.java:372)
at org.exist.start.Main.main(Main.java:48)
Caused by: org.apache.xmlrpc.XmlRpcException:
org.apache.xmlrpc.XmlRpcException: Invalid character data corresponding
to
XML entity 
at
org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java:457)
at
org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:163)
at
org.exist.xmldb.RemoteCollection.getResource(RemoteCollection.java:288)
2005-10-17
This week -end I switched from Mandriva to Fedora Core 4 .
08:32
What's interesting is that there is the latest gcj Java development kit
with
native eclipse . So I created a project from the eXist sources .
And guess what ?
It runs without problem? ;)
I will probably !
But just 6 compilation problems !
Severity Description
Resource In Folder
Location Creation
Time
2 The constructor
OutputStreamWriter(FileOutputStream, Charset) is
undefined DocumentView.java
eXist1.0/src/org/exist/client line
207 October 17, 2005 7:07:46 AM
2 The constructor
OutputStreamWriter(FileOutputStream, Charset) is
undefined InteractiveClient.java
eXist1.0/src/org/exist/client line
2428 October 17, 2005 7:07:46 AM
2 The import sun cannot be resolved
ForExpr.java
eXist1.0/src/org/exist/xquery line
39 October 17, 2005 7:07:24 AM
2 The method newDurationDayTime(boolean, BigInteger,
BigInteger,
BigInteger, BigDecimal) in the type DatatypeFactory is not applicable
for the
arguments (boolean, BigInteger, BigInteger, BigInteger,
BigInteger)
TimeUtils.java
eXist1.0/src/org/exist/xquery/value line
95 October
17, 2005 7:06:36 AM
2 Override cannot be resolved to a
type UntypedAtomicValue.java
eXist1.0/src/org/exist/xquery/value line
47 October 17, 2005 7:06:36
AM
2 Syntax error, annotations are only available if
source level is 5.0
UntypedAtomicValue.java
eXist1.0/src/org/exist/xquery/value line
47
October 17, 2005 7:06:36 AM
2005-09-26
Some job for eXist ?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=110157
2005-09-25
Refactoring : adding NodeProxy.TO_BE_COMPUTED in calls to
parentWithChild()
Good breakpoint for all cases :
PathExpr(AbstractExpression).eval(Sequence)
line: 43
Added support
for
HTTP URL's in fn:doc() . Works partly .....
Caused by: java.lang.ClassCastException: org.exist.memtree.DocumentImpl
at
org.exist.xquery.value.ValueSequence.toNodeSet(ValueSequence.java:163)
at
org.exist.memtree.NodeImpl.toNodeSet(NodeImpl.java:523)
at
org.exist.xquery.LocationStep.eval(LocationStep.java:154)
at
org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:43)
at org.exist.xquery.PathExpr.eval(PathExpr.java:159)
at
org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:43)
at org.exist.xquery.XQuery.execute(XQuery.java:136)
at
org.exist.xmldb.LocalXPathQueryService.execute(LocalXPathQueryService.java:330)
2005-09-24
I chase the -1 ;-)
DOCUMENT_NODE_ID = -1
In class NodeProxy: no javadoc on this :
What's the meaning of argument level == -1 ?
/* (non-Javadoc)
* @see
org.exist.dom.NodeSet#parentWithChild(org.exist.dom.DocumentImpl,
long, boolean, boolean, int)
*/
public NodeProxy parentWithChild(
DocumentImpl
otherDoc,
long otherId,
boolean
directParent,
boolean
includeSelf,
int level)
NodeProxy
Thread [Thread-7] (Suspended)
NodeProxy.getInternalAddress() line: 313
DOMFile.getNodeValue(NodeProxy, boolean) line: 1735
NativeBroker$7.start() line: 1619
NativeBroker$7(DOMTransaction).run() line: 56
NativeBroker.getNodeValue(NodeProxy, boolean) line: 1622
NodeProxy.getNodeValue() line: 289
NodeProxy.convertTo(int) line: 521
FunString.eval(Sequence, Item) line: 71
PathExpr.eval(Sequence, Item) line: 144
PathExpr(AbstractExpression).eval(Sequence) line: 43
XQuery.execute(CompiledXQuery, Sequence) line: 136
LocalXPathQueryService.execute(String[], NodeSet, CompiledExpression, String) line: 330
LocalXPathQueryService.execute(CompiledExpression) line: 131
QueryDialog$QueryThread.run() line: 397
The stack :
Thread [Thread-9] (Suspended)
DOMFile.findValue(Object,NodeProxy) line: 1087
DOMFile.getNodeValue(NodeProxy,boolean) line: 1737
NativeBroker$7.start()line: 1619
NativeBroker$7(DOMTransaction).run()line: 56
NativeBroker.getNodeValue(NodeProxy,boolean) line: 1622
NodeProxy.getNodeValue()line: 289
NodeProxy.convertTo(int)line: 521
FunString.eval(Sequence,Item) line: 71
PathExpr.eval(Sequence,Item) line: 144
PathExpr(AbstractExpression).eval(Sequence)line: 43
XQuery.execute(CompiledXQuery,Sequence) line: 136
LocalXPathQueryService.execute(String[],NodeSet,
CompiledExpression, String) line: 330
LocalXPathQueryService.execute(CompiledExpression)line:
131
QueryDialog$QueryThread.run()line: 397
2005-09-23
Misleading error message:
declare variable $workflows-conf as xs:string
{ document( concat($projectURI, '/workflow.xconf' ) ) };
entails:
Type error: the sequence cannot be converted into a node set.
Item type is xs:string
The exact contrary ! TODO check with latest CVS .
Another:
declare variable $workflows-conf as element()
{ document( concat($projectURI, '/workflow.xconf' )
) };
entails:
The type of variable $workflows-conf does not match the declared type:
element. Got: element.
2005-09-22
0. start server.sh
1. in the GUI create a new collection /db/projects
2. fill it with tens of files
3. look at
http://localhost:8080/exist/admin/admin.xql?panel=browse&collection=/db/projects
exception while retrieving collection: Collection
/db/projects not found
[at line 17, column 24]
4. restart server.sh
5. refresh
http://localhost:8080/exist/admin/admin.xql?panel=browse&collection=/db/projects
OK !?!
Not reproduced by Wolfgang and Giulio .
org.xmldb.api.base.XMLDBException
at
org.exist.xmldb.LocalXMLResource.getContent(LocalXMLResource.java:142)
at
org.exist.client.ClientFrame$TableMouseListener.mouseClicked(ClientFrame.java:1261)
at
java.awt.AWTEventMulticaster.mouseClicked(AWTEventMulticaster.java:212)
jmvanel At line 120 in process.xq
I propose instead of :
let $contains :=
for $ss in $list
return
if ( $ss = $s ) then true
else ""
===>
let $contains := some $ss in $list satisfies $ss eq $s
How do I get an XQuery call Stack ?
I see something in XPathException ...
-->| erichanson
(n=erichans@c-24-22-56-63.hsd1.or.comcast.net) has
joined #existdb
<--| erichanson has left
#existdb
jmvanel OK. I saw that it's done
in class FunctionCall .
I suppose that , if I don't get a call Stack, my error is not a
function
call, but at the root level ...
jmvanel Going out for a walk ...
wolf77 you can see a call stack if
the XPathException was created with
an XQueryAST node.
wolf77 i.e. new
XPathException(getASTNode(), ...)
wolf77 but some objects, like
values, don't have an AST node.
wolf77 In these cases, the
top-level expression can catch the
exception and add its own AST. This is what FunctionCall does.
jmvanel OK. The old code has not
new XPathException(getASTNode(),
...)
but it would be useful to add it when we come across it . Correct ?
wolf77 yes, exactly.
jmvanel And what about activating
the Profiler class ?
wolf77 You can activate it with an
option in the xquery:
wolf77 declare option exist:profiling
"enabled=yes verbosity=5";
Where could I look to solve this bug ?
let $v as element()* := ( <assign/> , <assign/> )
let $w := <r>{ $v }</r>
let $x as element()* := $w/assign
return $x
Error found: The type of variable $x does not match the declared type:
element. Got: element..
jmvanel_ suppress the 2nd as
element() and no error ...
jmvanel_ Also happens when $w is a
node from the DB .
2005-09-21
I'm doing Micheal Kay's tutorial with eXist:
http://www.stylusstudio.com/xquery_primer.html
doc('http://www.stylusstudio.com/examples/videos.xml')
Found 0 items. Compilation: 80ms, Execution: 7ms :-(
Conversation on #drools
conan exist is just two nots
jmvanel Sorry , two nots = ?
-->| mic_neale
(~mic_nea@ppp101-181.lns1.bne1.internode.on.net) has
joined #drools
conan yes
conan in Rete
conan an exist is actually two Nots
conan Not checks the the non
existence of a fact
conan so two nots, one after the
other, check for existence.
jmvanel The idea is to use drools
as a rewriter/optimiser for a
programming language.
The source code is avaiable as an AST (Abstract Syntaxic tree) of Java
objects .
conan ok
conan drools 3.0 would probably be
very usefull for that
conan especially the module and
truth maintenance
conan truth maintenace means that
one fact depends upon another.
conan so if a rule fires and that
consequence asserts a fact, that
fact can depend on the rule that fired it. if you remove the facts that
fired
the original rule it also removes the asserted facts.
conan modules give a kind of
controlled "flow".
conan each rule is associated with
a Module. the agenda, which
controlls the firing, has a stack of "modules". you set the current
module
which places it ontop of that stack. only rules in that current module
will
fire.
conan so you can have a load of
rules in a "validation" module and
another load of rles in a "process" module. Even though all rules might
be
true at the same time you can make sure all validation ones are
evaluated
first.
conan you will also be able to
check for the complete non existence of
a fact/condition in the WorkingMemory, or vice versa if a
fact/condition
exists.
jmvanel But without the thruth
maintenance (validation is allready
done), can't I use current Drools.
I have simple rules like this :
Source code:
/ root [ key = 123 ]
AND index present on key
==> rewrite as :
$key := util:qname-index-lookup( xs:QName("key"), 123 )
$root := $key / parent::root
conan jmvanel: sure if your rules
are simple enough you can use 2.0
2005-09-20
bug on satisfies
I investigate the bug on satisfies .
* Quantified expressions behave strangely
(Bug #1267554 - http://sourceforge.net/tracker/index.php?func=detail&aid=1267554&group_id=17691&atid=117691)
The evaluation of quantified expressions ('[some|every] $foo in [expr]
satisfies [expr]') seems to behave incorrectly. For example the
code
fragments
--------
every $foo in (1,2,3) satisfies
let $bar := baz
return false()
--------
and
--------
declare function local:foo() { false() };
every $bar in (1,2,3) satisfies
local:foo()
--------
both return true, whereas I would expect them to return false.
The original code in class QuantifiedExpression :
for(SequenceIterator i = inSeq.iterate(); i.hasNext(); ) {
contextItem = i.nextItem();
var.setValue(contextItem.toSequence());
var.checkType();
satisfiesSeq = returnExpr.eval(contextSequence);
// LOG.debug("satisfies returned: " + satisfiesSeq.getLength());
if(returnExpr.returnsType() == Type.BOOLEAN)
found = satisfiesSeq.effectiveBooleanValue();
else
found = satisfiesSeq.getLength() != 0;
if((mode == SOME && found) || (mode == EVERY && !found))
break;
}
context.popLocalVariables(mark);
return found ? BooleanValue.TRUE : BooleanValue.FALSE;
Proposed modification by Sam Stokes :
for(SequenceIterator i = inSeq.iterate(); i.hasNext(); ) {
contextItem = i.nextItem();
var.setValue(contextItem.toSequence());
var.checkType();
satisfiesSeq = returnExpr.eval(contextSequence);
// LOG.debug("satisfies returned: " + satisfiesSeq.getLength());
found = satisfiesSeq.effectiveBooleanValue();
if((mode == SOME && found) || (mode == EVERY && !found))
break;
}
context.popLocalVariables(mark);
return found ? BooleanValue.TRUE : BooleanValue.FALSE;
There is no test for the feature :
grep satisfies src/**/test/*.java
return nothing .
Debugging
every $foo in (1,2,3) satisfies
let $bar := baz
return false()
At first loop iteration :
returnExpr.returnsType()
(int) 11
i.e. Type.ITEM
satisfiesSeq.getLength()
(int) 1
satisfiesSeq.effectiveBooleanValue()
(boolean) false
satisfiesSeq is a ValueSequence whose value has type BooleanValue .
Code reading
Proposed code is simpler. It relies on function effectiveBooleanValue() that has the
desired behavior . Probably earlier the function
effectiveBooleanValue() t
was not reliable, so the extra test in current version. I think we can
accept
the patch.
bug on conversion to
boolean
Thread [main] (Suspended (breakpoint at line 185 in DoubleValue))
DoubleValue.effectiveBooleanValue() line: 185
FunBoolean.eval(Sequence, Item) line: 62
EnclosedExpr(PathExpr).eval(Sequence, Item) line: 144
EnclosedExpr.eval(Sequence, Item) line: 58
PathExpr.eval(Sequence, Item) line: 144
ElementConstructor.eval(Sequence, Item) line: 171
ElementConstructor(AbstractExpression).eval(Sequence) line: 43
PathExpr.eval(Sequence, Item) line: 159
PathExpr(AbstractExpression).eval(Sequence) line: 43
PathExpr.eval(Sequence, Item) line: 159
PathExpr(AbstractExpression).eval(Sequence) line: 43
LetExpr.eval(Sequence, Item, Sequence) line: 128
LetExpr(BindingExpression).eval(Sequence, Item) line: 110
LetExpr(AbstractExpression).eval(Sequence) line: 43
PathExpr.eval(Sequence, Item) line: 159
PathExpr(AbstractExpression).eval(Sequence) line: 43
XQuery.execute(CompiledXQuery, Sequence) line: 136
LocalXPathQueryService.execute(String[], NodeSet, CompiledExpression, String) line: 330
LocalXPathQueryService.doQuery(String, String[], NodeSet, String) line: 276
LocalXPathQueryService.query(String, String) line: 112
LocalXPathQueryService.query(String) line: 103
CopyOfXPathQueryTest.queryAndAssert(XQueryService, String, int, String) line: 147
CopyOfXPathQueryTest.testConvertToBoolean() line: 201
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
Method.invoke(Object, Object...) line: 585
CopyOfXPathQueryTest(TestCase).runTest() line: 154
CopyOfXPathQueryTest(TestCase).runBare() line: 127
TestResult$1.protect() line: 106
TestResult.runProtected(Test, Protectable) line: 124
TestResult.run(TestCase) line: 109
CopyOfXPathQueryTest(TestCase).run(TestResult) line: 118
TestSuite.runTest(Test, TestResult) line: 208
TestSuite.run(TestResult) line: 203
RemoteTestRunner.runTests(String[], String) line: 478
RemoteTestRunner.run() line: 344
RemoteTestRunner.main(String[]) line: 196
value == Double.NaN
is false !!!!!!!!!!!!!!!!! is value is a double equal to NaN .
$ grep --recursive xpath-functions src/
src/org/exist/xquery/Function.java:
"http://www.w3.org/2003/05/xpath-functions";
src/org/exist/xquery/functions/ModuleImpl.java: public final static
String
NAMESPACE_URI = "http://www.w3.org/2003/05/xpath-functions";
src/org/exist/xquery/Module.java:
"http://www.w3.org/2003/05/xpath-functions";
src/org/exist/util/Collations.java: public final
static String CODEPOINT =
"http://www.w3.org/2004/07/xpath-functions/collation/codepoint";
Unbound prefixes still not solved
!!!!!!!!!!!!!!!!!!!
I debugged :
/abc:*
when the prefix abc is not bound. Currently it behaves like /* .
A VirtualNodeSet is created
- this is normal
- shall we throw an error later when the VirtualNodeSet is
expanded ?
In fact at http://www.w3.org/TR/xquery/#node-tests
I got the answer about what to do :
So in XQueryContext there should be a way to check if a prefix has been
declared. When creating a NameTest in XQueryTreeParser the check can be
done
and the exception [err:XPST0008]
throwed
.
2005-09-26 :
still
not solved !!!!!!!!!!!!!!!!!!!
Tried to solve
XPathQueryTest.bugtestStarAxisConstraints()
but couldn't .
query:
/ * [ ./ * / t:title ]
J Revision Log
(Non-Normative) - in W3C Xquery specification
JMV: the red items
are to
check. The greens
are done . I
leave aside the Schema features.
This log records the changes that have been made to this document
since
the Last Call Draft of 12 November 2003.
J.1 23 July 2004
-
An ordering
declaration has been added to the
Prolog, which affects the ordering semantics of path expressions, FLWOR
expressions, and union
, intersect
, and except
expressions. In addition, ordered
and unordered
operators have been introduced that permit ordering semantics to be
controlled at the expression level within a query.
-
The SequenceType syntax has been simplified. SchemaContextPath
is no longer part of the SequenceType syntax. Also, the section on
SequenceType Matching has been rewritten to make its terminology more
consistent with the Formal Semantics document.
-
Validation has been separated from construction. Validation now
occurs only as a result of an explicit validate
expression. Validation modes are strict
and lax
,
and are specified on the validate
expression. New
construction modes strip
and preserve
have
been defined and are declared in the Prolog. The notion of "validation
context" has been deleted. The XQuery definition of validation has been
converged with the definition used in XSLT.
-
It is now permitted for multiple user-defined functions to have
the same name as long as they have different numbers of arguments.
-
The term xdt:untypedAny
is changed to xdt:untyped
.
-
xs:anyType
is no longer an abstract type, but is
now used to denote the type of a partially validated element node.
Since there is no longer a meaningful distinction between abstract
types and concrete types, these terms are no longer used in this
document.
-
Value comparisons now return ()
if either operand
is ()
.
-
Computed namespace constructors are now completely static and
are allowed only inside a computed element constructor. Namespace
declarations in a computed element constructor must come before the
element content, and must consist entirely of literals. The namespace
prefix is optional. If absent, it has the effect of setting the default
namespace for elements and types within the scope of the constructed
element.
-
The name of the Predicates grammar production has been changed
to PredicateList.
-
Grammar changes have been made in the Prolog, including
constraints on the order of declarations and imports.
-
The grammar production formerly called a FilterStep is now
called a FilterExpression and is described in a separate section rather
than as part of a path expression.
-
The precedence of the cast
and treat
operators in the grammar has been increased.
-
The precedence of unary arithmetic operators has been increased.
-
The syntax for variable
initialization in the Prolog now uses an assignment operator (":=
"). Also, circularities in
variable initialization are now considered to be a static errors.
-
The appendix containing a list of error messages has been made
non-normative.
-
An error is raised if a module attempts to import itself (target
namespace of importing module and imported modules are the same).
-
A schema can now be imported without specifying either a target
namespace or a location hint.
-
Module imports and schema imports now accept multiple location
hints, representing multiple physical resources in the same module or
schema.
-
CData Sections are no longer considered to be constructors, but
are simply a notational convenience for embedding special characters in
the content of an element or attribute constructor.
-
Three new components have been added to the static context:
XQuery Flagger status, XQuery Static Flagger status, and context item
static type. (Note: Flagger status items were later deleted.)
-
The expression data(())
no longer raises a static
error, although its static type is empty.
-
If a dynamic implementation detects a static error, the dynamic
evaluation phase is optional.
-
XPath allows host languages to specify whether they recognize
the concept of a namespace node. XQuery does not recognize namespace
nodes. Instead, it recognizes an "in-scope namespaces" property of an
element node.
-
The specification now clearly distinguishes between
"statically-known namespaces" (a static property of an expression) and
"in-scope namespaces" (a dynamic property of an element).
-
An order by
clause may now accept values of mixed
type if they have a common type that is reachable by numeric type
promotion and/or moving up the type derivation hierarchy, and if this
common type has a gt
operator.
-
In element and document node constructors, if the content
sequence contains a document node, that node is replaced by its
children (this was previously treated as an error).
-
Atomization now applies to the name expression of a computed
processing instruction constructor.
-
It is now implementation-defined whether undeclaration of
namespace prefixes in an element constructor is supported. If
supported, this feature conforms to the semantics of Namespaces 1.1. In
other words, if an element constructor binds a namespace prefix to the
zero-length string, any binding of that prefix defined at an outer
level is suspended within the scope of the constructed element.
-
The names of certain components in the static and dynamic
contexts have been changed to make them more consistent and descriptive.
-
In a computed text node constructor, the expression enclosed in
curly braces is no longer optional, since it is not possible to
construct an empty text node.
-
The definition of the idiv
operator has been made
consistent with the Functions and Operators document. The idiv
operator accepts operands of any numeric types, performs a division,
and truncates the result to an integer.
-
Certain grammar productions have been renamed and reordered to
make the grammar easier to read.
-
The section on constructor functions has been edited to make it
more consistent with the definition of constructor functions in the
Functions and Operators document.
-
The Comments section has been moved out of the Primary
Expressions section, because a comment is not a primary expression.
-
Rules for processing comment constructors have changed, to
ensure that the resulting comment does not contain adjacent hyphens or
end with a hyphen.
-
Some additional constraints have been added to the section
titled "Consistency Constraints".
-
Various minor changes, both editorial and substantive, have been
made in response to public comments and working group discussions.
J.2 29 Oct 2004
-
The last step in a path expression is now allowed to return a
sequence of atomic values or a sequence of nodes (mixed nodes and atomic values are not
allowed.)
-
A value of type xs:QName is now defined to consist of a
"triple": a namespace prefix, a namespace URI, and a local name.
Including the prefix as part of the QName value makes it possible to
cast any QName into a string when needed.
-
Local namespace declarations have been deleted from computed
element constructors. No namespace bindings may be declared by a
computed element constructors. Also, the body of a computed element
constructor has changed from ExprSingle to Expr.
-
The Prolog has been reorganized into three parts which must
appear in this order: (a) Setters; (b) Namespace declarations and
module and schema imports; (c) function and variable declarations.
-
A new "inherit-namespaces" declaration has been added to the
Prolog, and "namespace inheritance mode" has been added to the static
context.
-
An "encoding" subclause has been added to the Version
Declaration in the Prolog.
-
A new declaration has been added to the Prolog to control the
query-wide default handling of empty sequences in ordering keys ("empty
greatest" or "empty least".) "Default order for empty sequence" has
been added to static context.
-
XQuery Processing Model figure has been revised to include a
reference to modules.
-
The expression in the where-clause of a FLWOR expression has
changed from Expr to ExprSingle.
-
Clarification: The same variable name may be bound in more than
one for or let clause in a given FLWOR expression. This is not an
error, but each new binding occludes the previous one, which is no
longer accessible in the rest of the FLWOR expression.
-
In the static context, "current date" and "current time" have
been replaced by "current dateTime", which is defined to include a
timezone. Also, an implementation is required to initialize the current
dateTime and implicit timezone.
-
Computed comment constructors now raise an error rather than
trying to "fix up" an invalid comment by inserting blanks.
-
The div
operator can now be used to divide two
yearMonthDurations or two dayTimeDurations. In either case, the result
is of type xs:decimal
.
-
Additional signatures have been added to the fn:error
function.
-
The to
operator now accepts an empty sequence as
an operand, and returns empty sequence.
-
Support for XML 1.1 and Namespaces 1.1 have been bundled
together and defined as an optional feature. Various aspects of query
processing and serialization that depend on this optional feature have
been identified.
-
Descriptions of the arithmetic and logical operators have been
edited to eliminate dependencies on the order of operand evaluation.
-
The term "module resource," originally defined as a fragment of
XQuery code that can independently undergo static analysis, has been
replaced by the term "module."
-
Cyclic module imports are no longer permitted. A module M may
not import another module that directly or indirectly imports module M.
-
A new zero-argument version of fn:collection
has
been defined, to provide access to an implementation-dependent default
sequence.
-
A Mime encoding (application/xquery) has been defined for
XQuery, and a new Appendix has been added to describe it.
-
The operator mapping for numeric ge
and le
operators has been changed.
-
The section on typed values and string values has been edited to
clarify that an implementation may store either value and derive the
other. Also, an example has been added to illustrate the typed value of
a union type.
-
In a validate
expression, the node to be validated
is now mapped directly to an Infoset using the mapping defined in the
Data Model, rather than being serialized and parsed.
-
Various minor changes, both editorial and substantive, have been
made in response to public comments and working group discussions.
J.3 7 January 2005
-
A new section on Conformance has been added to the document.
-
The definition of Effective Boolean Value has changed.
-
The syntax for Pragmas has changed. The new syntax is described
in two new sections titled Extension Expressions and Option Declaration.
-
The XMLSpace declaration is now called a Boundary-space
declaration, to distinguish it from the unrelated attribute named
xml:space.
-
Changes have been made to the rules for namespace bindings in
element nodes that are copied by an element constructor. In the
grammar, "declare inherit-namespaces" has changed to "declare
copy-namespaces" with some new options.
-
Values of type xs:anyURI are now promotable to the type
xs:string, and are compared and ordered using the same default
collation used for strings.
-
A new symbol URILiteral has been added to the grammar and used
in places where a statically-known URI is required.
-
In a function declaration, if the declared function name has no
namespace prefix, it is now considered to be in the default namespace
for its module.
-
The XQuery Flagger and Static Flagger have been deleted from the
specification.
-
Constructor functions are now defined for types that are in no
namespace.
-
Type-related terms such as "schema type", "sequence type",
"static type", "dynamic type", and "type annotation" have been given
better definitions and more consistent usage.
-
Several error codes have been redefined or deleted.
-
Minor corrections have been made to the Operator Precedence
Table.
-
Various minor changes, both editorial and substantive, have been
made in response to public comments and working group discussions.
J.4 21 February 2005
-
Error codes have been expanded from six characters to eight
characters. Several error codes have been added, deleted, or modified.
-
New rules and examples have been added to the section on Errors
and Optimization.
-
A new consistency constraint has been added: the value of the
context item must match its static type.
-
Text has been added clarifying that, if the Static Typing
Feature is not in effect, expressions have an implementation-dependent
static type.
-
A rule has been added requiring an implementation to perform
whitespace normalization on URILiterals.
-
The grammatical rules governing the required order of items in
the Prolog have been changed.
-
The rule prohibiting a module from importing its own namespace
has been deleted.
-
The description of the Validate Expression has been reorganized
and rewritten.
-
Rules governing the baseURI properties of nodes created or
copied by constructors have changed.
-
The References section has been reorganized.
-
A new appendix has been added, enumerating all the
implementation-defined items.
-
Various minor changes, both editorial and substantive, have been
made in response to public comments and working group discussions.
J.5 04 April 2005
-
The keyword empty
has been changed to void
in the SequenceType grammar, in order to avoid conflict with the
built-in function named empty
.
-
Unordered mode now applies to sequences of atomic values
generated by the last step of a path expression, as well as node
sequences.
-
The section on constructor functions has been rewritten. The
semantics of constructor functions are now defined as equivalent to a cast
expression.
-
The definition of boundary
whitespace has been revised.
-
The section on validate
expressions now specifies
that the default validation mode is strict
(this default
was inadvertently omitted from earlier drafts.)
-
The version of Unicode that is supported by an XQuery
implementation is now implementation-defined.
-
The format in which errors are reported by an XQuery
implementation is now implementation-defined.
-
Various minor changes, both editorial and substantive, have been
made in response to public comments and working group discussions.
J Revision Log
(Non-Normative)
J.1 7 July 2005
-
An error has been corrected in the definition of the expansion
of leading-slash in a path expression.
-
Operators eq
and ne
are now defined
for the xs:duration
type.
-
Replaced reference to RFC2396 (URI's) with references to RFC3986
and RFC3987 (IRI's). Inserted text indicating that IRI's are accepted
where URI's are expected. Also replaced obsolete reference to RFC1738
by an updated reference to RFC3987.
-
Added text to section 2.5.2 clarifying that if the nilled
property of an element node is true, its typed value is the empty
sequence.
-
Added a consistency constraint stating that the "xml" prefix is
predefined, and it cannot be bound to anything else, and no other
prefix can be bound to the same URI.
-
In Section 3.7.1.3, if construction mode is strip
(Rule 1-e-ii-C-II), when stripping the type of an attribute: If the
name of the attribute is xml:ID, then the "is-ID" property of the
attribute is set to True; otherwise False.
-
Changed Section 3.7.1.3, Rule 1-e-ii-E. The base-URI property of
a copied node is no longer preserved, but inherited from the new
parent. Aligns XQuery with XSLT. New text for rule: "When an element or
processing instruction node is copied, its base-uri property is set to
be the same as that of its new parent, with the following exception: if
a copied element node has an xml:base attribute, its base-uri property
is set to the value of that attribute, resolved (if it is relative)
against the base-uri property of the new parent node. All other
properties of the copied nodes are preserved."
-
In Section 3.7.1.3, rules for copying element nodes when
construction mode is preserve
and copy-namespaces mode is
no-preserve
have been changed to introduce a new type
error: XQTY0086.
-
The string value of an empty direct constructor is now defined
as a zero-length string.
-
Some changes have been made in the default values of
serialization parameters.
-
Removed from the main part of the document any references to
line-ending normalization. Affects 3.1.1 (Literals) and 3.7.1.3 (Dir. Elem.
Constructor--Content, Rule 1a). Responds to Bug 1307.
Line ending normalization will be applied globally by the parser rather
than by individual expressions.
-
In SequenceType syntax, keyword void
is changed to
empty-sequence
.
-
Changes to function names: Changes function names: fn:subtract-dateTimes-yielding-dayTimeDuration
becomes op:subtract-dateTimes
and fn:subtract-dates-yielding-dayTimeDuration
becomes op:subtract-dates
(affects operator mapping
table).
J.2 15 September 2005
-
An error code (XQST0087) has been defined for an invalid
encoding specification in a Version Declaration.
-
Rules for processing the content of a direct element constructor
have been edited to clarify that boundary whitespace is processed
(possibly stripped) before entity and character references are expanded.
-
Rules for constructing
attributes with the name xml:id
have been modified. Additional
normalization rules apply to these attributes. The error
(XQST0082) that was raised when the value of an xml:id
attribute was not a valid NCName
has been deleted.
-
Error XPDY0044 is clarified to apply whenever the node-name
property of a constructed attribute node is xmlns
or xmlns:*
,
regardless of how the node-name
was computed.
-
The descriptions of all the node constructors have been changed
to specify that all newly-constructed nodes have an empty parent
property. The parent property of the new node can be set by an
outer-level constructor (if any).
-
The definition of a range expression has been edited to clarify
that, when the lower and upper bound of a range are the same integer,
only a single integer is returned by the range expression.
-
Values of numeric literals are now defined by the rules for
casting from xdt:untypedAtomic
into the appropriate
numeric type. This change defines overflow and underflow behavior for
numeric literals.
2005-09-17
Wolfgang corrected this :
declare variable $v as xs:string external;
this erased the value set by XQueryGenerator.
2005-09-16
http://cocoondev.org/main/117/42.html
another CMS
Suggestions:
- to debug an XQuery script, a function displaying the type
or an expression .
- Would that be difficult to implement the abiliy to
use an old eXist server with the newest client ?
jmvanel I can paste the two main
error stacks .
Are you aware of this project :
http://www.informatik.hu-berlin.de/mac/xquery/
The download is :-( not accessible .
A question about XQuery :
<a>{ $x, $y }</a>
and
<a>{ ( $x, $y ) }</a>
both work on eXist; are they both conforming to the spec ? |
|
wolf77_ |
I think yes, moment ... |
|
wolf77_ |
Yes, the EBNF defines
EnclosedExpr as: "{" Expr "}" and Expr is defined as: ExprSingle (","
ExprSingle)* |
|
jmvanel |
Maybe is it good practice
to use :
<a>{ ( $x, $y ) }</a>
to prepare the human reader for the following commas . |
|
jmvanel |
It the cost is the same ... |
|
wolf77_ |
yes, the cost should be
the same. |
I was surprised that :
let $w := <w initial="val" />
return <a>{ $w/@initial }</a>
returns
<a initial="val"/>
instead of:
<a>val</a>
but Saxon says the same .
* Would that be difficult
to implement the abiliy to use an old eXist server with the newest
client ?
I can paste the two main error stacks . |
|
wolf77 |
Yes, ok. Show me the stack
traces... |
|
jmvanel |
Caused by:
org.apache.xmlrpc.XmlRpcException:org.exist.EXistException:position
parameter out of bounds in call to retrieve: 0
at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java:457)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:163)
at org.exist.xmldb.RemoteXMLResource.getContent(RemoteXMLResource.java:158)
at org.exist.client.QueryDialog$QueryThread.run(QueryDialog.java:416)
Caused by: org.apache.xmlrpc.XmlRpcException:org.exist.EXistException:result
set unknown or timed out
at org.apache.xmlrpc.XmlRpcClient$Worker.execute(XmlRpcClient.java:457)
at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:163)
at org.exist.xmldb.RemoteXMLResource.getContent(RemoteXMLResource.java:158)
at org.exist.client.QueryDialog$QueryThread.run(QueryDialog.java:416)
|
|
jmvanel |
The second one appears in
loop after the first. |
|
wolf77 |
To fix the first one,
change line 155 in org.exist.xmldb.RemoteXMLResourceto |
|
wolf77 |
params.addElement(newInteger(pos
+ 1)); |
2005-08-01
Refactoring
Remains to do :
- add removeAttribute() to interface ContentLoadingObserver
(already in NativeValueIndexByQName)
- move this code in NativeBroker.endElement()
to NativeValueIndexByQName.endElement()
// TODO move_to NativeValueIndexByQName
if ( RangeIndexSpec.hasQNameIndex(indexType) ) {
if (content == null)
content = getNodeValue(tempProxy, false);
qnameValueIndex.endElement((ElementImpl) node, currentPath, content);
Guessing last
modification
time for an XQuery result
* the HTTP header Last-Modified is
filled with most recent time stamp
among all
* XQuery documents appearing in the
actual response.
* Note however, that the actual response
can be influenced, through
tests in the query,
* by documents more recent.
wget --output-document=- --save-headers \
'http://zamia:8080/exist/servlet/db/?_query=//property'
2005-07-31
Refactoring
Name changes (See ContentLoadingObserver)
:
addRow() --> markNode()
storeElement() --> markElement()
The relevant stack :
Thread [Thread-5] (Suspended)
NativeElementIndex.addRow(QName, NodeProxy) line: 99
NativeBroker.endElement(NodeImpl, NodePath, String) line: 2672
Indexer.endElement(String, String, String) line: 277
SAXParserImpl$JAXPSAXParser(AbstractSAXParser).endElement(QName, Augmentations) line: not available
XMLNSDocumentScannerImpl.scanEndElement() line: not available
XMLNSDocumentScannerImpl$NSContentDispatcher(XMLDocumentFragmentScannerImpl$FragmentContentDispatcher).dispatch(boolean) line: not available
XMLNSDocumentScannerImpl(XMLDocumentFragmentScannerImpl).scanDocument(boolean) line: not available
XIncludeAwareParserConfiguration(XML11Configuration).parse(boolean) line: not available
XIncludeAwareParserConfiguration(XML11Configuration).parse(XMLInputSource) line: not available
SAXParserImpl$JAXPSAXParser(XMLParser).parse(XMLInputSource) line: not available
SAXParserImpl$JAXPSAXParser(AbstractSAXParser).parse(InputSource) line: not available
SAXParserImpl$JAXPSAXParser.parse(InputSource) line: not available
Collection.store(Txn, DBBroker, IndexInfo, InputSource, boolean) line: 771
LocalCollection.storeXMLResource(LocalXMLResource) line: 642
LocalCollection.storeResource(Resource, Date, Date) line: 556
LocalCollection.storeResource(Resource) line: 548
InteractiveClient.parse(File[], UploadDialog) line: 1426
ClientFrame$37.run() line: 967
The sequence of calls in NativeBroker during storage of a
simple XML
document :
<a attr="aaa"><b>bbb</b></a>
Called (indirectly) by SAX parser :
doIndex()
case Node.ELEMENT_NODE
currentPath == /a
doIndex()
case Node.ATTRIBUTE_NODE
doIndex()
case Node.ELEMENT_NODE
currentPath == /a/b
doIndex()
case
Node.TEXT_NODE
endElement() currentPath
== /a/b content ==
null
endElement() currentPath
== /a content ==
null
Called by Collection.store() :
broker.storeDocument(transaction, document);
broker.closeDocument();
broker.flush();
Note: in Indexer.endElement() we have this dependancy on indexers :
XMLString elemContent = null;
if (GeneralRangeIndexSpec.hasQNameOrValueIndex(last.getIndexType())) {
elemContent = (XMLString) nodeContentStack.pop();
}
The above sequence of calls correspond to SAX
events; the callback functions for the indexers
(interface
ContentLoadingObserver ) could bear the same
names
as the SAX
events. This will make things clear for developpers.
However
ContentLoadingObserver.storeAttribute() has
no
corresponding method in SAX.
Call stack when removing a Document :
Thread [Thread-7] (Suspended)
NativeValueIndexByQName(NativeValueIndex).dropIndex(DocumentImpl) line: 336
NativeValueIndexByQName.dropIndex(DocumentImpl) line: 298
NativeBroker.notifyDropIndex(DocumentImpl) line: 339
NativeBroker.removeDocument(Txn, DocumentImpl, boolean) line: 2030
NativeBroker(DBBroker).removeDocument(Txn, DocumentImpl) line: 418
Collection.removeDocument(Txn, DBBroker, String) line: 677
LocalCollection.removeResource(Resource) line: 518
ClientFrame$33.run() line: 796
Thread.run() line: 595
Processing
XML with Java -
Welcome to Processing XML with Java , a
complete
tutorial about writing Java ...
It contains over 1000 pages of detailed information on SAX,
DOM,
JDOM,
2005-07-30
Discussion with
Perig
New features must be in permanent location.
I proposed this new Wiki page for rather "large" features, with several
days'
work .
Otherwise the pages at sf.net are to be used .
For example, in DBBroker (the *abstract* class for brokers), dataDir is
configured...
Perig_ It should be configured in
*native*Broker IMHO (because Native
= Local File system).
A classical Article about Spring and others : "Inversion of Control
Containers and the Dependency Injection pattern"
http://martinfowler.com/articles/injection.html
Nice thing is that there is allready an integration of
XML:DB
data sources in Spring :
http://Springxmldb.sf.net
But this is using eXist with Spring from the outside
. I think about
using the Spring facilities inside eXist .
Pas facile : comme cela a été
signalé dans la
liste, la notion de "journal" s'emmélait
joyeusement avec celle de
"log".
A propos, truc totalement
cosmétique :
DatableImpl, ligne 127 :
DatabaseImpl
System.out.println("configuring " + dbName + " using
" + home + '/' +
file);
Remplacer ce "/" par File.pathSeparator
Gaffe d'ailleurs : c'est souvent pathSeparatorChar
qui est
utilisé : les lecteurs réseau
de Windows risquent de ne pas
aimer.
Guessing last
modification time for an XQuery result
Is there some places in code where one can be pretty shure that a
database node has been really used in the ouput of the query ?
wolf77 Just at the very end,
before the query result is returned.
org/exist/xquery/XQuery.java
So in org/exist/xquery/XQuery.java
in theory I could do a recursive loop on
Sequence result
to examine each node's Document, but that might be expensive .
Either it would be an option, or above say, 100 nodes, do not output
the last
modification time .
wolf77 yes, that might be a good
solution. But you could just see how
much performance it costs and then decide.
Recovery
bugs
I have to re- read
http://wiki.exist-db.org/space/LoggingAndRecovery
and review the code .
I begin a comment on LoggingAndRecovery .
jmv@zamia: /usr/src/usr2/eXist-1.0 $ grep --recursive writeToLog src |
wc
79
256 7316
Refactoring
What is
planned
finish the refactoring in NativeBroker with interface
ContentLoadingObserver
refactor NativeValueIndex (900 lines) to make it look more like
NativeValueIndexByQName .
And remove calls deprecated from Indexable :
public byte[] serialize(short collectionId, boolean caseSensitive);
to be replaced by :
byte[] serializeValue (int offset, boolean caseSensitive);
Doing it
Wolf ? |
|
Still not modifying
NativeBroker ?
Before calling valueIndex.storeAttribute()
in NativeBroker.NodeProcessor.doIndex(),
currentPath.addComponent()is *not*
called
and before calling qnameValueIndex.storeAttribute()
currentPath.addComponent()*is*
called
What's the use of this call ? probably obsolete ? |
Began moving code from
NativeBroker to NativeValueIndexByQName
Before calling valueIndex.storeAttribute()
in NativeBroker.NodeProcessor.doIndex(),
currentPath.addComponent() is *not* called
and before calling qnameValueIndex.storeAttribute()
currentPath.addComponent() *is* called
What's the use of this call ? probably obsolete ?
|<-- Perig has left
irc.freenode.net (Read error: 54 (Connection
reset by peer))
jmvanel
Began moving code from NativeBroker to NativeValueIndexByQName
I'll commit , and then extract more code
Later I'll extract inner class NodeProcessor /
-->| Perig
(~chatzilla@vil35-2-82-236-174-106.fbx.proxad.net) has
joined #existdb
|wolf77| Calling
currentPath.addComponent() is certainly wrong. Either
it has to be called for all indexes or never. How does this look in the
original code?
jmvanel Let see in the past ...
Just before I introduced NativeValueIndexByQName
That is, may 25
revisoin 141
|wolf77| ok, in that revision,
addComponent() occurred at the very
beginning of the ATTRIBUTE case in method store().
that makes sense. So with the current code, searches
for attributes
should *fail*! Is this covered by the tests ...
No, it's not covered by ValueIndexTest. Could you
check that?
jmvanel Still trying to understand
what this call does ...
OK :
|wolf77| It's simple: in order to
check if a node has to be added to
an index, we need to compare the entire path leading to the node with
the
definitions in the configuration. NodePath is the class for storing the
path.
jmvanel currentPath passed in
argument doesn't include the current
node , right ?
|wolf77| it just contains the path
up to the parent element, yes.
There *is* a test at line 113 in ValueIndexTest : //item[@xx:test = 123]
But even if the value index is KO for the attributes
, the tests passes
.
|wolf77| yes, correct. However, we
don't know if the expression does
really use the value index. Have to look at the test output.
jmvanel Bacause the ValueIndex is
not testable .
If the the value index is OK the only difference is
that the test is
quicker . Right ?
|wolf77| Right. No, the log output
doesn't show it's using the
index.
In ValueIndexTest
vI get the message
The
specified
type: xs:double ... does not implement interface Indexable.
Concerning this attribute :
<price specialprice="true">1024.99</price>
With this thread :
Thread [main] (Suspended (breakpoint at line 723 in NativeValueIndex))
NativeValueIndex.convertToAtomic(int, String) line:
723
NativeValueIndex.storeAttribute(RangeIndexSpec,
AttrImpl) line: 143
NativeBroker$NodeProcessor.doIndex() line: 993
NativeBroker.store(Txn, NodeImpl, NodePath, boolean)
line: 2598
NativeBroker(DBBroker).store(Txn, NodeImpl,
NodePath) line: 470
Indexer.startElement(String, String, String,
Attributes) line: 561
SAXParserImpl$JAXPSAXParser(AbstractSAXParser).startElement(QName,
XMLAttributes, Augmentations) line: not available
XMLNSDocumentScannerImpl.scanStartElement() line:
not available
XMLNSDocumentScannerImpl$NSContentDispatcher(XMLDocumentFragmentScannerImpl$FragmentContentDispatcher).dispatch(boolean)
line: not available
XMLNSDocumentScannerImpl(XMLDocumentFragmentScannerImpl).scanDocument(boolean)
line: not available
XIncludeAwareParserConfiguration(XML11Configuration).parse(boolean)
line:
not available
XIncludeAwareParserConfiguration(XML11Configuration).parse(XMLInputSource)
line: not available
SAXParserImpl$JAXPSAXParser(XMLParser).parse(XMLInputSource) line: not
available
SAXParserImpl$JAXPSAXParser(AbstractSAXParser).parse(InputSource) line:
not available
SAXParserImpl$JAXPSAXParser.parse(InputSource) line:
not available
Collection.store(Txn, DBBroker, IndexInfo,
InputSource, boolean) line:
771
LocalCollection.storeXMLResource(LocalXMLResource)
line: 642
LocalCollection.storeResource(Resource, Date, Date)
line: 556
LocalCollection.storeResource(Resource) line: 548
ValueIndexTest.storeXMLFileAndGetQueryService(String, String) line:
177
ValueIndexTest.testStrings() line: 100
NativeMethodAccessorImpl.invoke0(Method, Object,
Object[]) line: not
available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[])
line: 39
DelegatingMethodAccessorImpl.invoke(Object,
Object[]) line: 25
Method.invoke(Object, Object...) line: 585
ValueIndexTest(TestCase).runTest() line: 154
ValueIndexTest(TestCase).runBare() line: 127
TestResult$1.protect() line: 106
TestResult.runProtected(Test, Protectable) line: 124
TestResult.run(TestCase) line: 109
ValueIndexTest(TestCase).run(TestResult) line: 118
TestSuite.runTest(Test, TestResult) line: 208
TestSuite.run(TestResult) line: 203
RemoteTestRunner.runTests(String[], String) line: 478
RemoteTestRunner.run() line: 344
RemoteTestRunner.main(String[]) line: 196
As Wolfgang wrote, I have to call
addComponent() at the very beginning of the ATTRIBUTE case
in
method doIndex(). It works .
I pass build.sh and I'll commit (19:37).
2005-07-29
Working on my client's xquery .
Added in class LocationPath a primitive XQuery profiler ; to
enable it
, change :
profilingEnabled = true ;
in org.exist.xquery/LocationStep.java
New XQuery function to return a date-valued HTTP header :
example :
let $datetime := '2005-07-22T22:22:22'
let $dummy := request:set-date-header( 'Last-Modified',
xs:dateTime($datetime) )
return $datetime
2005-07-28
While working on my client's xquery, I'm thinking of ways of attracting
new
users.
I thought that eXist must have a good stomach, i.e. able to absorb many
different foods.
I just added HTML input. Zip would not be difficult to add, it is
already on
the "catalog" of possible new new features.
There is an important type of data, that is not suitable for relational
databases: mail . If I could find an XML parser for mailbox, it would
be nice
to integrate it to eXist; But I can't find one ...
wolf77 hmmm, maybe through a
text2xml parser? I think I read about one
...
DiZzZz mailbox 2 xml..... hmmm
there a at least 2 formats. mbox and
....... the other one
jmvanel I found no javaCC grammar
for mail . What is name of the
concurrent to javaCC ?
wolf77 antlr?
jmvanel Yes.
Nothing for antlr .
Maybe there there is something to reuse from
javamail .
jmvanel There are "local store
providers" for JavaMail, supporting different Mail formats, e.g.
mstor.sf.net for mbox , or javamaildir.sf.net for qmail
format
.
2005-07-27
2005-07-25
Intesresting disussion with Perig . To really use eXist he would need a
spatial indexing.
http://www.geotools.org/Getting+Started
I see two solutions :
- develop a new special index mapping coordinates to
nodes,
and integrate geographic algo. to search the nearest
points
- integrate XQuark, and delegate the GIS part to
PostgreSQL or others
but there is some work in XQuark to pass the GIS SQL queries to
the XQuery
level
Added
http://wiki.exist-db.org/space/Possible+new+features:+a+catalog
system:ft-index-lookup(node(), xs:string)
A query like: /dblp/inproceedings[title &=
"sozial*"]
can be reformulated in different ways to use the
system:ft-index-lookup()
function. The most efficient formulation is:
would be:
/dblp/(system:ft-index-lookup(ancestor::title,
"sozial*")/parent::inproceedings)
twu This is called "Predicate Push
Down"?
wolf77 We don't have a name for it
yet ;-) "Selection Pull Up"?
twu I read about this technique
here:
http://www.ispras.ru/~grinev/mypapers/phd-short.pdf
Attribute push down is one technique to reduce element construction
(which is
costly).
The author also states that inlining of user defined
function is
important, since only then the whole query may be optimized.
Wolfgang , for the create index implementation,
we could have an iterator that would provide the
same events to
NativeBroker as the document loader, but by iterating on the DB or a
collection .
|
wolf77 |
Pooh, I finally merged
my NativeBroker with your changes. |
2005-07-24
the names of 4 classes
that read documents , and that could update a field latestTimeStamp in
the Context |
|
to output that in the HTTP
header Last-Modified .
|
|
wolf77 |
FunDoc, ExtCollection,
ExtDocument, RootNode |
|
As requested, I started to
write up some design notes here: http://wiki.exist-db.org/space/LoggingAndRecovery |
I implemented the last modification time column in the GUI : modifed
classes
:
- org/exist/client/InteractiveClient
- org/exist/client/ResourceDescriptor
- org/exist/client/ClientFrame
2005-07-23
Time 10:54
I updated my working dir. with Piotr's date code.
Need to commit the .classpath :
the Xerces files jar files have changed after Piotr's date code change.
I got rid of this frustrating little bug that is was here for weeks and
nobody cared . Out of 183 tests, just one failure !
[junit] org.exist.xquery.XPathException: cannot compare numeric value to non-numeric value
[junit] at org.exist.xquery.value.NumericValue.compareTo(NumericValue.java:74)
[junit] at org.exist.xquery.functions.FunDeepEqual.deepEquals(FunDeepEqual.java:72)
[junit] at org.exist.xquery.functions.FunDeepEqual.eval(FunDeepEqual.java:63)
[junit] at org.exist.xquery.PathExpr.eval(PathExpr.java:144)
[junit] at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:43)
[junit] at org.exist.xquery.XQuery.execute(XQuery.java:135)
[junit] at org.exist.xmldb.LocalXPathQueryService.execute(LocalXPathQueryService.java:330)
[junit] at org.exist.xmldb.LocalXPathQueryService.doQuery(LocalXPathQueryService.java:276)
[junit] at org.exist.xmldb.LocalXPathQueryService.query(LocalXPathQueryService.java:112)
[junit] at org.exist.xmldb.LocalXPathQueryService.query(LocalXPathQueryService.java:103)
[junit] at org.exist.xquery.test.DeepEqualTest.assertQuery(DeepEqualTest.java:248)
[junit] at org.exist.xquery.test.DeepEqualTest.testAtomic5(DeepEqualTest.java:50)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:585)
[junit] at junit.framework.TestCase.runTest(TestCase.java:154)
[junit] at junit.framework.TestCase.runBare(TestCase.java:127)
[junit] at junit.framework.TestResult$1.protect(TestResult.java:106)
[junit] at junit.framework.TestResult.runProtected(TestResult.java:124)
[junit] at junit.framework.TestResult.run(TestResult.java:109)
[junit] at junit.framework.TestCase.run(TestCase.java:118)
[junit] at junit.framework.TestSuite.runTest(TestSuite.java:208)
[junit] at junit.framework.TestSuite.run(TestSuite.java:203)
[junit] at junit.framework.TestSuite.runTest(TestSuite.java:208)
[junit] at junit.framework.TestSuite.run(TestSuite.java:203)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:326)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:560)
[junit] Loading mime table from file /usr/src/usr2/eXist-1.0/mime-types.xml
corresponding to the XQuery :
deep-equal(42, 'hello')
Inside this test in org.exist.xquery.functions.FunDeepEqual :
if (Type.subTypeOf(a.getType(), Type.ATOMIC) || Type.subTypeOf(b.getType(), Type.ATOMIC)) {
I replaced this
if (!Type.subTypeOf(a.getType(), Type.ATOMIC) ||
!Type.subTypeOf(b.getType(), Type.ATOMIC)) return false;
with this line , simpler and more general :
if ( a.getType() != b.getType() ) return false;
Time 15:05
It was not as simple as we thought ! First, the original logical test
was
correct .
But the function deep-equal() should not throw an error when atomic
types are
not compatible. The spec (http://www.w3.org/TR/xpath-functions/#func-deep-equal)
doesn't say so. Thus the function deep-equal() has a very different
behavior
compared to operator eq , which must
report
an error (http://www.w3.org/TR/xquery/#id-comparisons)
:
If the types of the operands, after evaluation, are not a valid
combination for the given operator, according to the rules in B.2 Operator Mapping,
a type error is
raised
[err:XPTY0004].
Trying with Saxon 8.4 from Saxonica
$ echo '111 eq "gggg"' > /tmp/bb.xq
$ xquery /tmp/bb.xq
Error
XPTY0004: Cannot compare xs:integer to xs:string
Failed to compile query
$ echo 'fn:deep-equal(111, "gggg")' > /tmp/bb.xq
$ xquery -wrap /tmp/bb.xq
<?xml version="1.0" encoding="UTF-8"?>
<result:sequence xmlns:result="http://saxon.sf.net/xquery-results" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<result:atomic-value xsi:type="xs:boolean">false</result:atomic-value>
Thus it's not good to call the algo. that is being used for the eq
operator
:
return GeneralComparison.compareAtomic( context.getDefaultCollator(),
(AtomicValue)a, (AtomicValue)b,
context.isBackwardsCompatible(), Constants.TRUNC_NONE, Constants.EQ );
Is there some other place in eXist where a tolerant comparison is
implemeted
or must I implement it from scratch ????
The Saxon mailing list:
http://sourceforge.net/mailarchive/forum.php?forum=saxon-help
No matches found for deep-equal
Time 15:57 :
With this implementation :
return a.atomize().compareTo(context.getDefaultCollator(), b.atomize()) == 0;
the new test atomic6 crashes :
assertQuery(true, "deep-equal( 1. , xs:integer(1) )" );
assertQuery(true, "deep-equal( xs:double(1) , xs:integer(1) )" );
true is the result of Saxon.
But with compareAtomic() all is OK :
return GeneralComparison.compareAtomic( context.getDefaultCollator(),
(AtomicValue)a, (AtomicValue)b,
context.isBackwardsCompatible(), Constants.TRUNC_NONE, Constants.EQ );
}
Wolgang collected some interesting links to small, but nice apps:
[junit] java.lang.Exception: Stack trace
[junit] at
java.lang.Thread.dumpStack(Thread.java:1158)
[junit] at
org.exist.storage.NativeBroker$9.start(NativeBroker.java:1803)
[junit] at
org.exist.storage.store.DOMTransaction.run(DOMTransaction.java:56)
[junit] at
org.exist.storage.NativeBroker.objectWith(NativeBroker.java:1819)
[junit] at
org.exist.dom.DocumentImpl.getFirstChild(DocumentImpl.java:386)
[junit] at
org.exist.storage.NativeBroker$11.start(NativeBroker.java:2001)
[junit] at
org.exist.storage.store.DOMTransaction.run(DOMTransaction.java:56)
[junit] at
org.exist.storage.NativeBroker.removeCollection(NativeBroker.java:2007)
[junit] at
org.exist.xmldb.LocalCollectionManagementService.removeCollection(LocalCollectionManagementService.java:130)
[junit] at
org.exist.xquery.test.DeepEqualTest.setupTestCollection(DeepEqualTest.java:269)
[junit] at
org.exist.xquery.test.DeepEqualTest.setUp(DeepEqualTest.java:283)
[junit] at
junit.framework.TestCase.runBare(TestCase.java:125)
[junit] at
junit.framework.TestResult$1.protect(TestResult.java:106)
[junit] at
junit.framework.TestResult.runProtected(TestResult.java:124)
[junit] at
junit.framework.TestResult.run(TestResult.java:109)
[junit] at
junit.framework.TestCase.run(TestCase.java:118)
[junit] at
junit.framework.TestSuite.runTest(TestSuite.java:208)
[junit] at
junit.framework.TestSuite.run(TestSuite.java:203)
[junit] at
junit.framework.TestSuite.runTest(TestSuite.java:208)
[junit] at
junit.framework.TestSuite.run(TestSuite.java:203)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:326)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:560)
2005-07-22
New article on eXist :
http://programming.newsforge.com/article.pl?sid=05/06/29/1429231&from=rss
I merge transaction development in NativeBroker and BrokerPool with my
refactorings in NativeBroker.
I found a problem (NPE) .
In NativeBroker line 1673 in method
public Collection getOrCreateCollection(String name)
final CollectionCache
collectionsCache =
pool.getCollectionsCache();
The NativeBroker is asking its BrokerPool for the collectionsCache .
But it
is null ! So the crash at next line.
The problem lies in the new method :
BrokerPool.initialize()
collectionsCache was was set after
the call to createBroker() . If I move the collectionsCache definition
just
before the call to createBroker() , all the tests pass
again .
Here is the interesting stack (i.e. not the stack at crash) :
Thread [main] (Suspended)
NativeBroker.<init>(BrokerPool, Configuration) line: 173
BrokerFactory.getInstance(BrokerPool, Configuration) line: 34
BrokerPool.createBroker() line: 444
BrokerPool.initialize() line: 566
BrokerPool.<init>(String, int, int, Configuration) line: 336
BrokerPool.configure(String, int, int, Configuration) line: 117
BrokerPool.configure(int, int, Configuration) line: 95
DOMIndexerTest.setUp() line: 167
DOMIndexerTest(TestCase).runBare() line: 125
TestResult$1.protect() line: 106
TestResult.runProtected(Test, Protectable) line: 124
TestResult.run(TestCase) line: 109
DOMIndexerTest(TestCase).run(TestResult) line: 118
TestSuite.runTest(Test, TestResult) line: 208
TestSuite.run(TestResult) line: 203
RemoteTestRunner.runTests(String[], String) line: 478
RemoteTestRunner.run() line: 344
RemoteTestRunner.main(String[]) line: 196
Suspect code :
NativeBroker: eclipse underlines that in yellow, because root is not
used
:
protected void repair() throws PermissionDeniedException {
Collection root = getCollection(ROOT_COLLECTION);
not shure readOnly is correct
2005-07-21
Questions on
transactions
- is there a way to avoid transmitting Transaction as an argument
in a lot of places ?
- how is a transaction started ?
- what is the starting point for Wolfgang transaction developments ?
NativeBroker started from this version :
Revision 1.154 - (view) (download) (annotate) - [select for diffs]
Sun Jun 12 19:06:322005 UTC (5 weeks, 2 days ago) by wolfgang_m
Branch: MAIN
- What is the difference between the class DOMTransaction and
the new transactions ?
Investigations
- There is very large number of modifications !
jmv@zamia: /usr/src/usr2/eXist-wolf-transac $ grep --recursive -i
--include='*.java' transact . | wc
1953
10773 215635
Integration of
transaction
code into CVS HEAD
I copied verbatim all this from Wolfgang's code :
storage/txn/
storage/log/
storage/recovery/
storage/lock/
EXistException.java
I did the minimal modifications here so that it compiles and runs (OK)
the
current CVS tests :
storage/BrokerPool.java
storage/DBBroker.java
storage/NativeBroker.java
Integration of new
transaction code for recovery
Dead lock with
test.concurrent.ConcurrentTestBase
Full thread dump Java HotSpot(TM) Client VM (1.5.0_03-b07 mixed mode):
"Thread-29" prio=1 tid=0x08324810 nid=0x5c77 waiting for monitor entry [0x8d01f000..0x8d01ff30]
at org.exist.storage.NativeBroker.openCollection(NativeBroker.java:482)
- waiting to lock <0x9178d000> (a org.exist.collections.CollectionCache)
at org.exist.storage.NativeBroker.getCollection(NativeBroker.java:451)
at org.exist.xmldb.LocalCollection.getCollection(LocalCollection.java:174)
at org.exist.xmldb.LocalCollection.<init>(LocalCollection.java:130)
at org.exist.xmldb.LocalCollection.<init>(LocalCollection.java:104)
at org.exist.xmldb.DatabaseImpl.getLocalCollection(DatabaseImpl.java:230)
at org.exist.xmldb.DatabaseImpl.getCollection(DatabaseImpl.java:161)
at org.xmldb.api.DatabaseManager.getCollection(Unknown Source)
at org.xmldb.api.DatabaseManager.getCollection(Unknown Source)
at org.exist.xmldb.test.concurrent.XQueryAction.execute(XQueryAction.java:57)
at org.exist.xmldb.test.concurrent.ConcurrentTestBase$Runner.run(ConcurrentTestBase.java:169)
"Thread-28" prio=1 tid=0x08324368 nid=0x5c76 waiting for monitor entry [0x8d0a0000..0x8d0a0fb0]
at org.exist.storage.NativeBroker.openCollection(NativeBroker.java:482)
- waiting to lock <0x9178d000> (a org.exist.collections.CollectionCache)
at org.exist.storage.NativeBroker.getCollection(NativeBroker.java:451)
at org.exist.xmldb.LocalCollection.getCollection(LocalCollection.java:174)
at org.exist.xmldb.LocalResourceSet.getResource(LocalResourceSet.java:157)
at org.exist.xmldb.test.concurrent.XQueryAction.execute(XQueryAction.java:71)
at org.exist.xmldb.test.concurrent.ConcurrentTestBase$Runner.run(ConcurrentTestBase.java:169)
"Thread-27" prio=1 tid=0x08326fe0 nid=0x5c75 in Object.wait() [0x8d833000..0x8d833e30]
at java.lang.Object.wait(Native Method)
- waiting on <0x91803f30> (a org.exist.util.ReentrantReadWriteLock)
at org.exist.util.ReentrantReadWriteLock.acquire(ReentrantReadWriteLock.java:62)
- locked <0x91803f30> (a org.exist.util.ReentrantReadWriteLock)
at org.exist.storage.NativeBroker.openCollection(NativeBroker.java:522)
- locked <0x9178d000> (a org.exist.collections.CollectionCache)
at org.exist.storage.NativeBroker.openCollection(NativeBroker.java:459)
at org.exist.xmldb.LocalCollection.getResource(LocalCollection.java:336)
at org.exist.xmldb.test.concurrent.RetrieveResourceAction.execute(RetrieveResourceAction.java:43)
at org.exist.xmldb.test.concurrent.ConcurrentTestBase$Runner.run(ConcurrentTestBase.java:169)
"Thread-26" prio=1 tid=0x08322008 nid=0x5c74 in Object.wait() [0x8d730000..0x8d730eb0]
at java.lang.Object.wait(Native Method)
- waiting on <0x917f5110> (a org.exist.xmldb.test.concurrent.ConcurrentTestBase$Runner)
at java.lang.Object.wait(Object.java:474)
at org.exist.util.MultiReadReentrantLock.writeLock(MultiReadReentrantLock.java:161)
- locked <0x917f5110> (a org.exist.xmldb.test.concurrent.ConcurrentTestBase$Runner)
at org.exist.util.MultiReadReentrantLock.acquire(MultiReadReentrantLock.java:89)
at org.exist.collections.Collection.checkPermissions(Collection.java:1273)
at org.exist.collections.Collection.determineTreeStructure(Collection.java:1115)
at org.exist.collections.Collection.validate(Collection.java:863)
at org.exist.xmldb.LocalCollection.storeXMLResource(LocalCollection.java:601)
at org.exist.xmldb.LocalCollection.storeResource(LocalCollection.java:538)
at org.exist.xmldb.LocalCollection.storeResource(LocalCollection.java:530)
at org.exist.xmldb.test.concurrent.DBUtils.addXMLResource(DBUtils.java:122)
at org.exist.xmldb.test.concurrent.ReplaceResourceAction.execute(ReplaceResourceAction.java:79)
at org.exist.xmldb.test.concurrent.ConcurrentTestBase$Runner.run(ConcurrentTestBase.java:169)
"Thread-25" prio=1 tid=0x08437658 nid=0x5c6b in Object.wait() [0x8d7b1000..0x8d7b2130]
at java.lang.Object.wait(Native Method)
- waiting on <0x9178eec8> (a org.exist.storage.sync.SyncDaemon)
at org.exist.storage.sync.SyncDaemon.nextTask(SyncDaemon.java:345)
- locked <0x9178eec8> (a org.exist.storage.sync.SyncDaemon)
at org.exist.storage.sync.SyncDaemon$RunLoop.run(SyncDaemon.java:380)
at java.lang.Thread.run(Thread.java:595)
"Low Memory Detector" daemon prio=1 tid=0x080dd378 nid=0x5a54 runnable [0x00000000..0x00000000]
"CompilerThread0" daemon prio=1 tid=0x080dbe28 nid=0x5a53 waiting on condition [0x00000000..0x8ea74a98]
"Signal Dispatcher" daemon prio=1 tid=0x080daf58 nid=0x5a52 waiting on condition [0x00000000..0x00000000]
"Finalizer" daemon prio=1 tid=0x080cfbe0 nid=0x5a51 in Object.wait() [0x8eba9000..0x8eba9fb0]
at java.lang.Object.wait(Native Method)
- waiting on <0x915a6a48> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0x915a6a48> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Reference Handler" daemon prio=1 tid=0x080ceee0 nid=0x5a50 in Object.wait() [0x8ec2a000..0x8ec2ae30]
at java.lang.Object.wait(Native Method)
- waiting on <0x915a6ac8> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:474)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0x915a6ac8> (a java.lang.ref.Reference$Lock)
"main" prio=1 tid=0x0805bdf8 nid=0x5a4e in Object.wait() [0xbfffb000..0xbfffc998]
at java.lang.Object.wait(Native Method)
- waiting on <0x917f5110> (a org.exist.xmldb.test.concurrent.ConcurrentTestBase$Runner)
at java.lang.Thread.join(Thread.java:1095)
- locked <0x917f5110> (a org.exist.xmldb.test.concurrent.ConcurrentTestBase$Runner)
at java.lang.Thread.join(Thread.java:1148)
at org.exist.xmldb.test.concurrent.ConcurrentTestBase.testConcurrent(ConcurrentTestBase.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at junit.framework.TestCase.runTest(TestCase.java:154)
at junit.framework.TestCase.runBare(TestCase.java:127)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:326)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.executeInVM(JUnitTask.java:990)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.execute(JUnitTask.java:662)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.execute(JUnitTask.java:636)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:302)
at org.apache.tools.ant.Task.perform(Task.java:401)
at org.apache.tools.ant.Target.execute(Target.java:338)
at org.apache.tools.ant.Target.performTasks(Target.java:365)
at org.apache.tools.ant.Project.executeTarget(Project.java:1237)
at org.apache.tools.ant.Project.executeTargets(Project.java:1094)
at org.apache.tools.ant.Main.runBuild(Main.java:669)
at org.apache.tools.ant.Main.startAnt(Main.java:220)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:215)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:90)
"VM Thread" prio=1 tid=0x080cc2f8 nid=0x5a4f runnable
"VM Periodic Task Thread" prio=1 tid=0x080de808 nid=0x5a55 waiting on condition
2005-07-20
merge [ options ] file1 file2 file3
DESCRIPTION
merge incorporates
all changes that lead from file2 to file3
into
file1. The result ordinarily
goes into file1.
-p
Send results to standard output instead of overwriting
file1.
So I did :
cd /usr/src/usr2/eXist-wolf-transac/
cp NativeBroker.java NativeBroker-wolf.java
echo ':ext:jmvanel@cvs.sourceforge.net:/cvsroot/exist' > CVS/Root
cvs update -r 1.154 NativeBroker.java
merge -p $exist/src/org/exist/storage/NativeBroker.java \
NativeBroker.java NativeBroker-wolf.java \
> NativeBroker-merge.java
merge: warning: conflicts during merge
merge -p NativeBroker-wolf.java \
NativeBroker.java \
$exist/src/org/exist/storage/NativeBroker.java \
> NativeBroker-merge2.java
http://xml.sys-con.com/read/104980.htm Indexer.storeElement(): why
pushing an
empty XMLString The pushed XMLString is used to collect character data
while
parsing the element content.
Contents