OcamlParser - Revision history

Mike: add to contributor category

2018-01-31T11:03:27Z

add to contributor category

← Older revision		Revision as of 11:03, 31 January 2018
Line 38:		Line 38:
	A good example of how to use the tree iteration and transformation modules can be found in {{srcbranch\|hugo\|ocaml/infer_derivatives.ml}}. This syntax transformer receives makes <code>DERIVATIVE OF</code> declarations optional by inferring them from the uses of the <code>der()</code> operator throughout the program.		A good example of how to use the tree iteration and transformation modules can be found in {{srcbranch\|hugo\|ocaml/infer_derivatives.ml}}. This syntax transformer receives makes <code>DERIVATIVE OF</code> declarations optional by inferring them from the uses of the <code>der()</code> operator throughout the program.

	[[Category:GSOC2016]] [[Category:Experimental]]		[[Category:GSOC2016]] [[Category:Experimental]][[Category:ASCEND Contributors]]

Hugo at 16:49, 22 August 2016

2016-08-22T16:49:39Z

← Older revision		Revision as of 16:49, 22 August 2016
Line 3:		Line 3:
	The new parser was designed to make it easy to parse ASCEND syntax into a syntax tree, perform transformation passes over this syntax tree and then serialize the result back into a textual format, which can be fed into the existing ASCEND compiler. This lets us implement new language features in terms of existing ones and experiment without needing to modify the core ASCEND compiler.		The new parser was designed to make it easy to parse ASCEND syntax into a syntax tree, perform transformation passes over this syntax tree and then serialize the result back into a textual format, which can be fed into the existing ASCEND compiler. This lets us implement new language features in terms of existing ones and experiment without needing to modify the core ASCEND compiler.

	One advantage of having this new parser and syntax-tree tooling is that working with syntax trees is hard to do in the existing parsing architecture because ASCEND's C compiler parses and interprets the code at the same time, and doesn't give us a chance to apply syntax transformers before the code is interpreted. Another advantage of the new parser is that it is written in Ocaml, which as a functional programming language is specially suited for operating on tree-like data structures.		One advantage of having this new parser and syntax-tree tooling is that working with syntax trees is hard to do in the existing parsing architecture because ASCEND's C compiler parses and interprets the code at the same time, and doesn't give us a chance to apply syntax transformers before the code is interpreted. Another advantage of the new parser is that it is written in [https://ocaml.org/ Ocaml], which as a functional programming language is specially suited for operating on tree-like data structures.

	== Compiling and Using the New Parser ==		== Compiling and Using the New Parser ==
Line 28:		Line 28:
	</source>		</source>

	The makefile shows how to compile the code and the tests filder has some examples of how to run it. For more details see {{srcbranch\|hugo\|ocaml/README.txt}}.		The makefile, {{srcbranch\|hugo\|ocaml/makefile}}, shows how to compile the code and the tests filder has some examples of how to run it. For more details see {{srcbranch\|hugo\|ocaml/README.txt}}.

	== Writing Syntax Transformers ==		== Writing Syntax Transformers ==

	The goal of this new Ocaml parser is to support syntax experiments that work by mapping ASCEND syntax trees (possibly with some extensions) into a vanilla ASCEND syntax tree that can be fed into the existing C compiler. However, writing these syntax tree transformers by hand can be a very laborious process because you need to write a lot of code just to traverse the syntax tree and get to the interesting leaves that we might want to modify. (Have a look at tokenize_tree to see how traversing a tree by hand looks like).		The goal of this new Ocaml parser is to support syntax experiments that work by mapping ASCEND syntax trees (possibly with some extensions) into a vanilla ASCEND syntax tree that can be fed into the existing C compiler. However, writing these syntax tree transformers by hand can be a very laborious process because you need to write a lot of code just to traverse the syntax tree and get to the interesting leaves that we might want to modify. (Have a look at {{srcbranch\|hugo\|ocaml/tokenize_tree.ml}} to see how traversing a tree by hand looks like).

	One of the ways to avoid boilerplate traversal code is to use the "Visitor Pattern" and write traversals by extending a default traversal that doesn't do anything. To support this pattern, there is an interface for syntax tree iterators in ast_iterator.ml as well as a default_iterator that doesn't do anything but that can be easily extended into an iterator that does something interesting in relevant nodes and just walks through everything else. There is also a similar system for transformers (functions that convert one AST into another) in ast_mapper.ml.		One of the ways to avoid boilerplate traversal code is to use the "Visitor Pattern" and write traversals by extending a default traversal that doesn't do anything. To support this pattern, there is an interface for syntax tree iterators in {{srcbranch\|hugo\|ocaml/ast_iterator.ml}} as well as a default_iterator that doesn't do anything but that can be easily extended into an iterator that does something interesting in relevant nodes and just walks through everything else. There is also a similar system for transformers (functions that convert one AST into another) in {{srcbranch\|hugo\|ocaml/ast_mapper.ml}}.

	A good example of how to use the tree iteration and transformation modules can be found in infer_derivatives.ml. This syntax transformer receives makes <code>DERIVATIVE OF</code> declarations optional by inferring them from the uses of the <code>der()</code> operator throughout the program.		A good example of how to use the tree iteration and transformation modules can be found in {{srcbranch\|hugo\|ocaml/infer_derivatives.ml}}. This syntax transformer receives makes <code>DERIVATIVE OF</code> declarations optional by inferring them from the uses of the <code>der()</code> operator throughout the program.

	[[Category:GSOC2016]] [[Category:Experimental]]		[[Category:GSOC2016]] [[Category:Experimental]]

Hugo at 08:23, 22 August 2016

2016-08-22T08:23:53Z

← Older revision		Revision as of 08:23, 22 August 2016
Line 3:		Line 3:
	The new parser was designed to make it easy to parse ASCEND syntax into a syntax tree, perform transformation passes over this syntax tree and then serialize the result back into a textual format, which can be fed into the existing ASCEND compiler. This lets us implement new language features in terms of existing ones and experiment without needing to modify the core ASCEND compiler.		The new parser was designed to make it easy to parse ASCEND syntax into a syntax tree, perform transformation passes over this syntax tree and then serialize the result back into a textual format, which can be fed into the existing ASCEND compiler. This lets us implement new language features in terms of existing ones and experiment without needing to modify the core ASCEND compiler.

	One advantage of having this new parser and syntax-tree tooling is that working with syntax trees is hard to do in the existing parsing architecture because ASCEND's C compiler parses and interprets the code at the same time, and doesn't give us a chance to apply syntax transformers before the code is interpreted. Another advantage of the new parser is that it is written in ~~[https://ocaml.org/~~ Ocaml], which as a functional programming language is specially suited for operating on tree-like data structures.		One advantage of having this new parser and syntax-tree tooling is that working with syntax trees is hard to do in the existing parsing architecture because ASCEND's C compiler parses and interprets the code at the same time, and doesn't give us a chance to apply syntax transformers before the code is interpreted. Another advantage of the new parser is that it is written in Ocaml, which as a functional programming language is specially suited for operating on tree-like data structures.

	== Compiling and Using the New Parser ==		== Compiling and Using the New Parser ==
Line 12:		Line 12:
	svn co http://svn.ascend4.org/branches/hugo ascend-hugo</nowiki>		svn co http://svn.ascend4.org/branches/hugo ascend-hugo</nowiki>

	To run the new parser and to write syntax transformers, you will need to install the Ocaml compiler, the [https://opam.ocaml.org/ Opam] package manager and ~~some additional dependencies~~. ~~On Ubuntu (or Debian?) this is achieved as follows:~~		To run the new parser and to write syntax transformers, you will need to install the Ocaml compiler, and some Ocaml libraries. The recommended way to do this is to use your distro's package manager to install the Ocaml compiler and the [https://opam.ocaml.org/ Opam] package manager and then use Opam to download and install the required Ocaml libraries.

	<source lang=sh>		<source lang=sh>
	~~# Install Ocaml and the Opam package manager~~		sudo apt-get install ocaml opam
	sudo apt-install ocaml opam

	~~# Install Ocaml dependencies (locally)~~		opam init
	opam install menhir containers sequence		opam install menhir containers sequence
	</source>		</source>
Line 29:		Line 28:
	</source>		</source>

	The makefile~~, {{srcbranch\|hugo\|ocaml/makefile}},~~ shows how to compile the code and the tests filder has some examples of how to run it. For more details see {{srcbranch\|hugo\|ocaml/README.txt}}.		The makefile shows how to compile the code and the tests filder has some examples of how to run it. For more details see {{srcbranch\|hugo\|ocaml/README.txt}}.

	== Writing Syntax Transformers ==		== Writing Syntax Transformers ==

	The goal of this new Ocaml parser is to support syntax experiments that work by mapping ASCEND syntax trees (possibly with some extensions) into a vanilla ASCEND syntax tree that can be fed into the existing C compiler. However, writing these syntax tree transformers by hand can be a very laborious process because you need to write a lot of code just to traverse the syntax tree and get to the interesting leaves that we might want to modify. (Have a look at ~~{{srcbranch\|hugo\|ocaml/~~tokenize_tree~~.ml}}~~ to see how traversing a tree by hand looks like).		The goal of this new Ocaml parser is to support syntax experiments that work by mapping ASCEND syntax trees (possibly with some extensions) into a vanilla ASCEND syntax tree that can be fed into the existing C compiler. However, writing these syntax tree transformers by hand can be a very laborious process because you need to write a lot of code just to traverse the syntax tree and get to the interesting leaves that we might want to modify. (Have a look at tokenize_tree to see how traversing a tree by hand looks like).

	One of the ways to avoid boilerplate traversal code is to use the "Visitor Pattern" and write traversals by extending a default traversal that doesn't do anything. To support this pattern, there is an interface for syntax tree iterators in ~~{{srcbranch\|hugo\|ocaml/~~ast_iterator.ml}} as well as a default_iterator that doesn't do anything but that can be easily extended into an iterator that does something interesting in relevant nodes and just walks through everything else. There is also a similar system for transformers (functions that convert one AST into another) in ~~{{srcbranch\|hugo\|ocaml/~~ast_mapper.ml}}.		One of the ways to avoid boilerplate traversal code is to use the "Visitor Pattern" and write traversals by extending a default traversal that doesn't do anything. To support this pattern, there is an interface for syntax tree iterators in ast_iterator.ml as well as a default_iterator that doesn't do anything but that can be easily extended into an iterator that does something interesting in relevant nodes and just walks through everything else. There is also a similar system for transformers (functions that convert one AST into another) in ast_mapper.ml.

	A good example of how to use the tree iteration and transformation modules can be found in ~~{{srcbranch\|hugo\|ocaml/~~infer_derivatives.ml}}. This syntax transformer receives makes <code>DERIVATIVE OF</code> declarations optional by inferring them from the uses of the <code>der()</code> operator throughout the program.		A good example of how to use the tree iteration and transformation modules can be found in infer_derivatives.ml. This syntax transformer receives makes <code>DERIVATIVE OF</code> declarations optional by inferring them from the uses of the <code>der()</code> operator throughout the program.

	[[Category:GSOC2016]] [[Category:Experimental]]		[[Category:GSOC2016]] [[Category:Experimental]]

Jpye: /* Compiling and Using the New Parser */

2016-08-22T08:00:02Z

Compiling and Using the New Parser

← Older revision		Revision as of 08:00, 22 August 2016
Line 29:		Line 29:
	</source>		</source>

	The {{srcbranch\|hugo\|ocaml/makefile}} shows how to compile the code and the tests filder has some examples of how to run it. For more details see {{srcbranch\|hugo\|ocaml/README.txt}}.		The makefile, {{srcbranch\|hugo\|ocaml/makefile}}, shows how to compile the code and the tests filder has some examples of how to run it. For more details see {{srcbranch\|hugo\|ocaml/README.txt}}.

	== Writing Syntax Transformers ==		== Writing Syntax Transformers ==

Jpye at 07:59, 22 August 2016

2016-08-22T07:59:42Z

← Older revision		Revision as of 07:59, 22 August 2016
Line 29:		Line 29:
	</source>		</source>

	The makefile shows how to compile the code and the tests filder has some examples of how to run it. For more details see {{srcbranch\|hugo\|ocaml/README.txt}}.		The {{srcbranch\|hugo\|ocaml/makefile}} shows how to compile the code and the tests filder has some examples of how to run it. For more details see {{srcbranch\|hugo\|ocaml/README.txt}}.

	== Writing Syntax Transformers ==		== Writing Syntax Transformers ==

Jpye at 07:58, 22 August 2016

2016-08-22T07:58:20Z

← Older revision		Revision as of 07:58, 22 August 2016
Line 3:		Line 3:
	The new parser was designed to make it easy to parse ASCEND syntax into a syntax tree, perform transformation passes over this syntax tree and then serialize the result back into a textual format, which can be fed into the existing ASCEND compiler. This lets us implement new language features in terms of existing ones and experiment without needing to modify the core ASCEND compiler.		The new parser was designed to make it easy to parse ASCEND syntax into a syntax tree, perform transformation passes over this syntax tree and then serialize the result back into a textual format, which can be fed into the existing ASCEND compiler. This lets us implement new language features in terms of existing ones and experiment without needing to modify the core ASCEND compiler.

	One advantage of having this new parser and syntax-tree tooling is that working with syntax trees is hard to do in the existing parsing architecture because ASCEND's C compiler parses and interprets the code at the same time, and doesn't give us a chance to apply syntax transformers before the code is interpreted. Another advantage of the new parser is that it is written in Ocaml, which as a functional programming language is specially suited for operating on tree-like data structures.		One advantage of having this new parser and syntax-tree tooling is that working with syntax trees is hard to do in the existing parsing architecture because ASCEND's C compiler parses and interprets the code at the same time, and doesn't give us a chance to apply syntax transformers before the code is interpreted. Another advantage of the new parser is that it is written in [https://ocaml.org/ Ocaml], which as a functional programming language is specially suited for operating on tree-like data structures.

	== Compiling and Using the New Parser ==		== Compiling and Using the New Parser ==

Jpye: /* Writing Syntax Transformers */ activate links

2016-08-22T07:56:50Z

Writing Syntax Transformers: activate links

← Older revision		Revision as of 07:56, 22 August 2016
Line 33:		Line 33:
	== Writing Syntax Transformers ==		== Writing Syntax Transformers ==

	The goal of this new Ocaml parser is to support syntax experiments that work by mapping ASCEND syntax trees (possibly with some extensions) into a vanilla ASCEND syntax tree that can be fed into the existing C compiler. However, writing these syntax tree transformers by hand can be a very laborious process because you need to write a lot of code just to traverse the syntax tree and get to the interesting leaves that we might want to modify. (Have a look at tokenize_tree to see how traversing a tree by hand looks like).		The goal of this new Ocaml parser is to support syntax experiments that work by mapping ASCEND syntax trees (possibly with some extensions) into a vanilla ASCEND syntax tree that can be fed into the existing C compiler. However, writing these syntax tree transformers by hand can be a very laborious process because you need to write a lot of code just to traverse the syntax tree and get to the interesting leaves that we might want to modify. (Have a look at {{srcbranch\|hugo\|ocaml/tokenize_tree.ml}} to see how traversing a tree by hand looks like).

	One of the ways to avoid boilerplate traversal code is to use the "Visitor Pattern" and write traversals by extending a default traversal that doesn't do anything. To support this pattern, there is an interface for syntax tree iterators in ast_iterator.ml as well as a default_iterator that doesn't do anything but that can be easily extended into an iterator that does something interesting in relevant nodes and just walks through everything else. There is also a similar system for transformers (functions that convert one AST into another) in ast_mapper.ml.		One of the ways to avoid boilerplate traversal code is to use the "Visitor Pattern" and write traversals by extending a default traversal that doesn't do anything. To support this pattern, there is an interface for syntax tree iterators in {{srcbranch\|hugo\|ocaml/ast_iterator.ml}} as well as a default_iterator that doesn't do anything but that can be easily extended into an iterator that does something interesting in relevant nodes and just walks through everything else. There is also a similar system for transformers (functions that convert one AST into another) in {{srcbranch\|hugo\|ocaml/ast_mapper.ml}}.

	A good example of how to use the tree iteration and transformation modules can be found in infer_derivatives.ml. This syntax transformer receives makes <code>DERIVATIVE OF</code> declarations optional by inferring them from the uses of the <code>der()</code> operator throughout the program.		A good example of how to use the tree iteration and transformation modules can be found in {{srcbranch\|hugo\|ocaml/infer_derivatives.ml}}. This syntax transformer receives makes <code>DERIVATIVE OF</code> declarations optional by inferring them from the uses of the <code>der()</code> operator throughout the program.

	[[Category:GSOC2016]] [[Category:Experimental]]		[[Category:GSOC2016]] [[Category:Experimental]]

Jpye: /* Compiling and Using the New Parser */

2016-08-22T07:54:49Z

Compiling and Using the New Parser

← Older revision		Revision as of 07:54, 22 August 2016
Line 7:		Line 7:
	== Compiling and Using the New Parser ==		== Compiling and Using the New Parser ==

	The code for the new experimental parser as well as the infrastructure for writing syntax transformers can currently be found in the "hugo" branch of the ASCEND Subversion repository. It has not been merged into the trunk yet.		The code for the new experimental parser as well as the infrastructure for writing syntax transformers can currently be found in the {{srcbranchdir\|hugo\|}} branch of the ASCEND Subversion repository. It has not been merged into the trunk yet.

	<nowiki>		<nowiki>
	svn ~~checkout~~ http://svn.ascend4.org/branches/hugo ascend-hugo</nowiki>		svn co http://svn.ascend4.org/branches/hugo ascend-hugo</nowiki>

	To run the new parser and to write syntax transformers, you will need to install the Ocaml compiler, the [https://opam.ocaml.org/ Opam] package manager and some additional dependencies.		To run the new parser and to write syntax transformers, you will need to install the Ocaml compiler, the [https://opam.ocaml.org/ Opam] package manager and some additional dependencies. On Ubuntu (or Debian?) this is achieved as follows:

	<~~nowiki~~>		<source lang=sh>
	# Install Ocaml and the Opam package manager		# Install Ocaml and the Opam package manager
	sudo apt-install ocaml opam		sudo apt-install ocaml opam

	# Install Ocaml dependencies (locally)		# Install Ocaml dependencies (locally)
	opam install menhir containers sequence</~~nowiki~~>		opam install menhir containers sequence
			</source>

	Then, go to the "ocaml" folder in the ASCEND source directory and build things with the makefile. Now you can start playing with the new parser.		Then, go to the "ocaml" folder in the ASCEND source directory and build things with the makefile. Now you can start playing with the new parser.

	<~~nowiki~~>		<source lang=sh>
	cd ascend-hugo		cd ascend-hugo/ocaml
	cd ocaml		make
	make</~~nowiki~~>		</source>

	The makefile shows how to compile the code and the tests filder has some examples of how to run it. For more details see ~~the~~ README.txt ~~in the ocaml folder~~.		The makefile shows how to compile the code and the tests filder has some examples of how to run it. For more details see {{srcbranch\|hugo\|ocaml/README.txt}}.

	== Writing Syntax Transformers ==		== Writing Syntax Transformers ==

Hugo: Created page with "For GSOC2016, Hugo implemented an experimental parser for the ASCEND modeling language that allows a faster rate of experimentation for new syntactic feature..."

2016-08-22T04:50:39Z

Created page with "For GSOC2016, Hugo implemented an experimental parser for the ASCEND modeling language that allows a faster rate of experimentation for new syntactic feature..."

New page

For [[GSOC2016]], [[User:Hugo|Hugo]] implemented an experimental parser for the ASCEND modeling language that allows a faster rate of experimentation for new syntactic features compared to the existing ASCEND parser.

The new parser was designed to make it easy to parse ASCEND syntax into a syntax tree, perform transformation passes over this syntax tree and then serialize the result back into a textual format, which can be fed into the existing ASCEND compiler. This lets us implement new language features in terms of existing ones and experiment without needing to modify the core ASCEND compiler.

One advantage of having this new parser and syntax-tree tooling is that working with syntax trees is hard to do in the existing parsing architecture because ASCEND's C compiler parses and interprets the code at the same time, and doesn't give us a chance to apply syntax transformers before the code is interpreted. Another advantage of the new parser is that it is written in Ocaml, which as a functional programming language is specially suited for operating on tree-like data structures.

== Compiling and Using the New Parser ==

The code for the new experimental parser as well as the infrastructure for writing syntax transformers can currently be found in the "hugo" branch of the ASCEND Subversion repository. It has not been merged into the trunk yet.

<nowiki>
svn checkout http://svn.ascend4.org/branches/hugo ascend-hugo</nowiki>

To run the new parser and to write syntax transformers, you will need to install the Ocaml compiler, the [https://opam.ocaml.org/ Opam] package manager and some additional dependencies.

<nowiki>
# Install Ocaml and the Opam package manager
sudo apt-install ocaml opam

# Install Ocaml dependencies (locally)
opam install menhir containers sequence</nowiki>

Then, go to the "ocaml" folder in the ASCEND source directory and build things with the makefile. Now you can start playing with the new parser.

<nowiki>
cd ascend-hugo
cd ocaml
make</nowiki>

The makefile shows how to compile the code and the tests filder has some examples of how to run it. For more details see the README.txt in the ocaml folder.

== Writing Syntax Transformers ==

The goal of this new Ocaml parser is to support syntax experiments that work by mapping ASCEND syntax trees (possibly with some extensions) into a vanilla ASCEND syntax tree that can be fed into the existing C compiler. However, writing these syntax tree transformers by hand can be a very laborious process because you need to write a lot of code just to traverse the syntax tree and get to the interesting leaves that we might want to modify. (Have a look at tokenize_tree to see how traversing a tree by hand looks like).

One of the ways to avoid boilerplate traversal code is to use the "Visitor Pattern" and write traversals by extending a default traversal that doesn't do anything. To support this pattern, there is an interface for syntax tree iterators in ast_iterator.ml as well as a default_iterator that doesn't do anything but that can be easily extended into an iterator that does something interesting in relevant nodes and just walks through everything else. There is also a similar system for transformers (functions that convert one AST into another) in ast_mapper.ml.

A good example of how to use the tree iteration and transformation modules can be found in infer_derivatives.ml. This syntax transformer receives makes <code>DERIVATIVE OF</code> declarations optional by inferring them from the uses of the <code>der()</code> operator throughout the program.

[[Category:GSOC2016]] [[Category:Experimental]]