Doc updates

README.rst

@@ -94,7 +94,7 @@ History
        - Implemented Eclipse OMR JIT backend
        - Created `Ravi with batteries <https://github.com/dibyendumajumdar/Suravi>`_.
 * 2019 
-       - New parser, type checker for Ravi - work in progress
+       - `New parser, type checker for Ravi <https://the-ravi-programming-language.readthedocs.io/en/latest/ravi-new-parser-codegenerator.html>`_ - work in progress
        - New language feature - `defer` statement
        - New JIT backend `MIR <https://github.com/vnmakarov/mir>`_. 
 

readthedocs/ravi-new-parser-codegenerator.rst

@@ -1,19 +1,75 @@
 New Parser and Code Generator for Ravi
 ======================================
 
+Status
+------
+The basic parser and type checker have been implemented. However the ``defer`` statement is not yet done.
+Progress is very slow but things are moving every now and then when I get time.
+
+Usage
+-----
+The parse tree can be generated as follows::
+
+   tree, errmessage = ast.parse 'some lua code'
+   if not tree then
+      print(errmessage)
+   else
+      print(tree:tostring())
+   end
+
+Examples::
+
+   tree, errmessage = ast.parse 'print "hello"'
+   if not tree then
+      print(errmessage)
+   else
+      print(tree:tostring())
+   end
+   
+This generates output::
+
+  function()
+    --[expression statement start]
+     --[expression list start]
+       --[suffixed expr start] any
+        --[primary start] any
+          print --global symbol ?
+        --[primary end]
+        --[suffix list start]
+          --[function call start] any
+           (
+             'hello'
+           )
+          --[function call end]
+        --[suffix list end]
+       --[suffixed expr end]
+     --[expression list end]
+    --[expression statement end]
+  end
+
+For more examples please see:
+
+* `ravi-tests/ravi_test_ast.lua <https://github.com/dibyendumajumdar/ravi/blob/master/ravi-tests/ravi_test_ast.lua>`_ 
+* `ravi-tests/ravi_test_ast2.lua <https://github.com/dibyendumajumdar/ravi/blob/master/ravi-tests/ravi_test_ast2.lua>`_. 
+
+If you run these scripts the parse AST will be dumped to stdout.
+
+Design Notes
+------------
+
 These are some design notes on the new parser / code generator.
 
 There will be several phases:
 
 1. Convert input into syntax tree - this will look very much like the input source, i.e. there will be fornum loops, if statements,
-   while and repeat loops etc. During this phase only the types for literals and local decalarations will be known.
+   while and repeat loops etc. During this phase only the types for literals and local decalarations will be known. MOSTLY DONE.
 2. The next phase will be a type checking phase. In this phase types will be derive for expressions, and also type assertions will be
    added where required such as unpon function entry, and after function calls. But the overall syntax tree will still resemble the input 
-   source except for the additional instructions.
+   source except for the additional instructions. MOSTLY DONE. 
 3. Third phase will be to assign virtual registers; first to locals, and then to temporaries. For simplicity I will probably keep the
    temporaries and locals in separate ranges.
 4. Next phase will be linearize the instructions - during this phase the loops, if statements etc will get translated to equivalent of 
-   conditional and unconditional jumps. 
+   conditional and unconditional jumps. We will probably create intermediate structures that can easily map to Ravi bytecodes.
 5. In the first phase we will stop here, generate byte code and finish. 
 6. The next phase will be translate into basic blocks.
 7. Following that we will construct a CFG, perform dominator analysis and convert to SSA form.
@@ -28,8 +84,11 @@ Some other things
    to generate more optimized code.
 
 
-Current Status
+Implementation
 --------------
-We have a parser implementation that can convert Ravi source to an abstract syntax tree (AST). Static type checking is being worked on. Progress is very slow but things are moving every now and then when I get time.
+The new Ravi Parser and Code Generator implementation is in:
+
+* `ravi_ast_parse.c <https://github.com/dibyendumajumdar/ravi/blob/master/src/ravi_ast_parse.c>`_ - contains the parser that builds AST
+* `ravi_ast_print.c <https://github.com/dibyendumajumdar/ravi/blob/master/src/ravi_ast_print.c>`_ - contains utilities for printing out the AST
+* `ravi_ast_typecheck.c <https://github.com/dibyendumajumdar/ravi/blob/master/src/ravi_ast_typecheck.c>`_ - contains the type checking phase of the parser
 
-For examples of how to call the parser please see `ravi-tests/ravi_test_ast.lua <https://github.com/dibyendumajumdar/ravi/blob/master/ravi-tests/ravi_test_ast.lua>`_ and `ravi-tests\ravi_test_ast2.lua <https://github.com/dibyendumajumdar/ravi/blob/master/ravi-tests/ravi_test_ast2.lua>`_. If you run these scripts the parse AST will be dumped to stdout.