adding_keywords

If changing grammar files (.g), we need to be able to run the Antlr parser-generator. We use Ant to do the build, but Ant has to be able to find Antlr. Choose Window > Preferences > Ant > Runtime > Classpath. Select Global Entries. Click Add Jars..., and select org.prorefactor.lib/antlr/antlr.jar file. Click Add Variable... and enter ${workspace_loc:/org.prorefactor.core/bin/}. This is required because there is additional post-processing which must be done after Antlr has generated the source for the tree parser. You can close this Preferences window now.

In the package which contains the tree parser, select the build.xml file. Right-click, and choose Run > Ant Build... to get the properties dialog.

Choose the Refresh tab, and enable the automatic refresh of the containing folder. Now when you choose Run > Ant Build, the refresh and re-compile will be done automatically.

The following steps are necessary when adding keywords:

• Add node type to BaseTokenTypes.txt
• add keywords to proparse.g reserved or unreserved list
• build and compile com.joanju.proparse.
• Add node type to com.joanju.proparse.NodeTypes
• Copy ProParserTokenTypes.txt to org.prorefactor.treeparserbase and org.prorefactor.treeparser01.
• Touch the tree parsers' .g file before doing a new build, or it won't read the updated TokenTypes.txt.
• Build the two tree parsers.
• Update org.prorefactor.core.TokenTypes.java.

Note that JPTreeParser.g is in two folders: treeparserbase and treeparser01. It is a direct copy. The treeparser01.g inherits from JPTreeParser.g. I edit/build treeparserbase first, then copy JPTreeParser.g to treeparser01 before I start work on edit/build in that folder. You can create your own tree parser, inheriting from JPTreeParser.g. It's a little complicated, but very powerful once you figure out what Antlr is doing here.

Have a look at /org.prorefactor.core/src/org/prorefactor/core/unittest/TestNewSyntax.java:

 /** Test all tree parsers against new syntax.
   * These tests just run the tree parsers against the data/newsyntax directory.
   * If no exceptions are thrown, then the tests pass.
   * The files in the "newsyntax" directories are subject to change, so no other
   * tests should be added other than the expectation that they parse clean.
   */

You can see the schema and propath for that project in prorefactor/projects/unittest. It's the sports2000 schema, with the "data" directory in the propath.

You should be able to add new .p, .w, or .cls files to data/newsyntax and the unit test should find them and test that they parse OK. Throw in a bit of invalid syntax to check that the unittest fails as expected. Or, better yet, add an ABL file with new (failing) syntax first before updating Proparse so that you can see the unit tests go from "fail" to "pass".