Get Started¶
Introduction¶
The ASTx library is an agnostic framework for constructing and representing Abstract Syntax Trees (ASTs). Its primary objective is to provide a versatile and language-independent structure for ASTs, with the flexibility to be utilized across various programming languages and parsing tools. ASTx doesn't aim to be a lexer or a parser, although it could be used by any programming language or parser in order to provide a high level representation of the AST.
Many kinds of nodes (classes) are currently supported. Below is a list with just some examples:
Statements:¶
- Function
- Function Prototype
- FunctionReturn
- ForRangeLoop
- VarDecl
Operators:¶
- BinaryOp
- UnaryOp
Data types:¶
- Boolean
- Literal
- Variable
The ASTx project is still under development, so new classes may be added to the ones above at any time.
Below are installation instructions and an example, so you can have an overview of how you can leverage the ASTx library for your needs.
Installation¶
The first step is to install ASTx. You can do it simply by running the command below in your terminal:
$ pip install astx
If you need more information on installation, you can get it in the ASTx installation page.
After that, you can just open a Jupyter Notebook instance and start writing your first AST.
Example: an AST of a series of mathematical operations¶
Here we will present a quick example of an AST of the expression
basic_op = lit_1 + b - a * c / a + (b - a / a), in which
$~~~~$ lit_1 is a defined integer, and
$~~~~$ a, b, and c are variables.
The first thing to do is, in your Jupyter Notebook instance, import display, which will allow you to have a basic visualization of the AST, and the astx library itself.
In [1]:
Copied!
# import display for AST visualization
from IPython.display import display
from astx.viz import traverse_ast_ascii, graph_to_ascii
import astx
# import display for AST visualization
from IPython.display import display
from astx.viz import traverse_ast_ascii, graph_to_ascii
import astx
Then we create an instance of the Module class, and this instance will be the first node of the tree, or the root node. After that, we declare the variables and literal that will be part of the basic operation that we will parse into an AST.
In [2]:
Copied!
# Create module
module = astx.Module()
# Declare variables
decl_a = astx.VariableDeclaration(name="a", type_=astx.Int32, value=astx.LiteralInt32(1))
decl_b = astx.VariableDeclaration(name="b", type_=astx.Int32, value=astx.LiteralInt32(2))
decl_c = astx.VariableDeclaration(name="c", type_=astx.Int32, value=astx.LiteralInt32(4))
a = astx.Variable(name="a")
b = astx.Variable(name="b")
c = astx.Variable(name="c")
# Declare literal
lit_1 = astx.LiteralInt32(1)
# State the expression
basic_op = lit_1 + b - a * c / a + (b - a / a)
# Create module
module = astx.Module()
# Declare variables
decl_a = astx.VariableDeclaration(name="a", type_=astx.Int32, value=astx.LiteralInt32(1))
decl_b = astx.VariableDeclaration(name="b", type_=astx.Int32, value=astx.LiteralInt32(2))
decl_c = astx.VariableDeclaration(name="c", type_=astx.Int32, value=astx.LiteralInt32(4))
a = astx.Variable(name="a")
b = astx.Variable(name="b")
c = astx.Variable(name="c")
# Declare literal
lit_1 = astx.LiteralInt32(1)
# State the expression
basic_op = lit_1 + b - a * c / a + (b - a / a)
After the basic expression is stated, we create an instance of the Function class. As mentioned in the API documentation, each instance of the Function class must have a prototype and a body, so we'll create those first.
The body is made of a block that is created and the variables, as well as the basic operation, are appended to it afterwards.
In [3]:
Copied!
# Create FunctionPrototype
main_proto = astx.FunctionPrototype(
name="main", args=astx.Arguments(), return_type=astx.Int32
)
# Create FunctionReturn
main_block = astx.Block()
main_block.append(decl_a)
main_block.append(decl_b)
main_block.append(decl_c)
main_block.append(astx.FunctionReturn(basic_op))
# Create Function
main_fn = astx.Function(prototype=main_proto, body=main_block)
# Append function to module
module.block.append(main_fn)
# Create FunctionPrototype
main_proto = astx.FunctionPrototype(
name="main", args=astx.Arguments(), return_type=astx.Int32
)
# Create FunctionReturn
main_block = astx.Block()
main_block.append(decl_a)
main_block.append(decl_b)
main_block.append(decl_c)
main_block.append(astx.FunctionReturn(basic_op))
# Create Function
main_fn = astx.Function(prototype=main_proto, body=main_block)
# Append function to module
module.block.append(main_fn)
After this, the module is complete. We can get its AST structure as a dictionary, as well as a PNG representation.
In [4]:
Copied!
# Create dictionary representation
module.get_struct()
# Create dictionary representation
module.get_struct()
Out[4]:
{'MODULE[main]': {'content': [{'FUNCTION[main]': {'content': {'args': {'Arguments(0)': {'content': [],
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.GenericKind: -100>}}},
'body': {'BLOCK': {'content': [{'VariableDeclaration[a, Int32]': {'content': {'Literal[Int32]: 1': {'content': 1,
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '57dfddcd509a40f386e5a6b39859c751',
'kind': <ASTKind.GenericKind: -100>}}},
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.VarDeclKind: -203>}}},
{'VariableDeclaration[b, Int32]': {'content': {'Literal[Int32]: 2': {'content': 2,
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': 'cb308f1b129642efb97476bef4db4299',
'kind': <ASTKind.GenericKind: -100>}}},
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.VarDeclKind: -203>}}},
{'VariableDeclaration[c, Int32]': {'content': {'Literal[Int32]: 4': {'content': 4,
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '98b423efcd3542668e8d2ebaf4c45d06',
'kind': <ASTKind.GenericKind: -100>}}},
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.VarDeclKind: -203>}}},
{'RETURN': {'content': {'BINARY[+]': {'content': {'lhs': {'BINARY[-]': {'content': {'lhs': {'BINARY[+]': {'content': {'lhs': {'Literal[Int32]: 1': {'content': 1,
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': 'f35a559775db447b821e933ba025cbb8',
'kind': <ASTKind.GenericKind: -100>}}},
'rhs': {'Variable[b]': {'content': 'b',
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.GenericKind: -100>}}}},
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.BinaryOpKind: -301>}}},
'rhs': {'BINARY[/]': {'content': {'lhs': {'BINARY[*]': {'content': {'lhs': {'Variable[a]': {'content': 'a',
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.GenericKind: -100>}}},
'rhs': {'Variable[c]': {'content': 'c',
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.GenericKind: -100>}}}},
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.BinaryOpKind: -301>}}},
'rhs': {'Variable[a]': {'content': 'a',
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.GenericKind: -100>}}}},
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.BinaryOpKind: -301>}}}},
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.BinaryOpKind: -301>}}},
'rhs': {'BINARY[-]': {'content': {'lhs': {'Variable[b]': {'content': 'b',
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.GenericKind: -100>}}},
'rhs': {'BINARY[/]': {'content': {'lhs': {'Variable[a]': {'content': 'a',
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.GenericKind: -100>}}},
'rhs': {'Variable[a]': {'content': 'a',
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.GenericKind: -100>}}}},
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.BinaryOpKind: -301>}}}},
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.BinaryOpKind: -301>}}}},
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.BinaryOpKind: -301>}}},
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.ReturnKind: -403>}}}],
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.GenericKind: -100>}}}},
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.FunctionKind: -401>}}}],
'metadata': {'loc': {line: -1, col: -1},
'comment': '',
'ref': '',
'kind': <ASTKind.ModuleKind: -101>}}}
In [5]:
Copied!
# Create ascii representation
dot_graph = traverse_ast_ascii(module.get_struct(simplified=True))
graph = graph_to_ascii(dot_graph)
print(graph)
# Create ascii representation
dot_graph = traverse_ast_ascii(module.get_struct(simplified=True))
graph = graph_to_ascii(dot_graph)
print(graph)
┌────────────┐
│MODULE[main]│
└──────┬─────┘
│
v
┌──────────────┐
│FUNCTION[main]│
└────┬────┬────┘
│ │
┌───────────────────────┘ └────────────────────────┐
│ │
v │
┌───────┐ │
│ body │ │
└───┬───┘ │
│ │
v │
┌─────────┐ │
│ BLOCK │ │
└─┬───┬┬┬─┘ │
│ │││ │
┌───────────────────────┘ ││└──────────────────────────────┐ │
│ ┌────────────────────┘│ │ │
v │ │ │ │
┌───────┐ │ │ │ │
│ RETURN│ │ │ │ │
└───┬───┘ │ │ │ │
│ │ │ │ │
┌──────────┼──────┼─────────────────────┘ │ │
┌──────┘ │ │ │ │
│ v │ │ │
│ ┌─────────┐ │ │ │ ┌─────────┐
│ │BINARY[+]│ │ │ │ │BINARY[+]│
│ └───┬──┬──┘ │ │ │ └──┬─┬────┘
│ │ │ │ │ │ │ │ ^
│ │ │ │ │ └──────────────┐ │ │ │
│ │ │ │ │ └───┐ │ │ │
│ │ └────────┼──────────────────────────────────────────────┐ │ │ │ │ │
│ └───────────┼────────────────────┐ │ │ │ │ │ │
│ │ │ ┌──────────────────────┼──────┼──────────────────────────────────────┼───┘ │ │
│ │ │ │ │ ┌───┼──────────────────────────────────────┼─────┘ │
│ │ │ │ │ │ │ │ │
│ │ v v v v │ │ │
│ │ ┌───────┐ ┌───────┐ │ │ │
│ │ │ lhs │ │ rhs │ │ │ │
│ │ └──┬──┬─┘ └──┬──┬─┘ │ │ │
│ │ │ │ │ │ │ │ │
│ │ ┌────────────┘ └────────────┐ │ │ └────────────────────┐ │ │
│ │ │ │ │ └────────────┐ └──┐ │ │
│ ┌─────────────┘ │ │ │ │ │ │ │
│ ┌─┘ │ │ │ │ │ │ │
│ │ v │ v │ │ │ │
│ │ ┌─────────┐ │ ┌─────────┐ │ │ │ │
│ │ │BINARY[-]│ │ │BINARY[-]│ │ │ │ │
│ │ └───┬──┬──┘ │ └───┬──┬──┘ │ │ │ │
│ │ │ │ │ │ │ │ │ │ │
│ │ │ └────────┐ │ │ └─────┐ │ │ │ │
│ │ │ │ ┌──────────────┼─────┘ │ │ │ │ │
│ │ │ │ │ │ │ │ │ │ │
│ │ │ v v │ │ │ │ │ │
│ │ │ ┌───────┐ │ │ │ │ │ │
│ │ │ │ rhs │ │ │ │ │ │ │
│ │ │ └───┬──┬┘ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ ┌───────┼──────────────┼───────┘
│ │ │ v v │ │ │ │ │ │
│ │ │ ┌─────────┐ │ │ │ │ │ │
│ │ │ │BINARY[/]│ │ │ │ │ │ │
│ │ │ └─┬─┬─┬─┬─┘ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │ │ │ │ │
│ │ │ ┌──────┘ │ │ │ │ │ │ │ │ │
│ │ │ │ ┌─────┘ │ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │ └──────┼┐ │ │
│ │ │ │ │ │ │ │ │ │└──┐ │ │
│ │ │ │ │ │ │ └───────────┐ │ │ │ │ │
│ │ │ │ │ │ └───────────────────┐ │ │ │ │ │ │
│ │ │ │ │ └───────────┐ │ │ │ │ │ │ │
│ │ └───┼──┼───────────────────┼─────────┼────┼──┼─────────────┐ │ │ │ │
│ │ v v │ │ │ │ │ │ │ │ │
│ │ ┌───────┐ │ │ │ │ │ │ │ │ │
│ │ │ lhs │ │ │ │ │ │ │ │ │ │
│ │ └┬────┬─┘ │ │ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │ │ │ │ └──────────────────┐
│ │ │ │ │ │ │ │ │ │ │ └─────┐ │
│ │ │ │ │ │ │ └───────────┐ │ │ │ │ │
│ │ │ │ │ │ └────────────┐ │ │ │ │ │ │
│ │ │ └───────────────────┼─────────┼─────┐ │ │ │ │ │ │ │
└─────┐ │ │ │ │ │ │ │ │ │ │ │ │
│ │ v │ │ │ │ │ │ │ │ │ │
│ │ ┌─────────┐ │ │ │ │ │ │ │ │ │ │
│ │ │BINARY[*]│ │ │ │ │ │ │ │ │ │ │
│ │ └───┬──┬──┘ │ │ │ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ └─┼─┐ │ │ │ │
│ │ │ │ │ └─────┼───────┐ │ │ │ │ │ │ │
│ │ │ │ └───────────────┼────┐ │ │ │ │ │ │ │ │
│ │ │ └─────────────────────────────────┐ │ │ │ │ │ │ │ │ │ │
│ │ └──────────────────────────┐ │ │ │ │ │ │ │ │ │ │ │
│ └────────────────────────────┐ │ │ │ │ │ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │ │ │ │ │ │
v v v v │ v v │ v v │ │ v v
┌─────────────────────────────┐ ┌─────────────────────────────┐ ┌───────┐ ┌───────┐ │ ┌───────┐ │ ┌─────┴─┐ │ ┌─────────────────────────────┐ ┌───────┐
│VariableDeclaration[c, Int32]│ │VariableDeclaration[b, Int32]│ │ lhs │ │ rhs │ │ │ rhs │ │ │ lhs │ │ │VariableDeclaration[a, Int32]│ │ args │
└────────┬────────────────────┘ └──────────────┬──────────────┘ └───┬───┘ └───┬───┘ │ └──┬──┬─┘ │ └───┬───┘ │ └────────────────────────────┬┘ └───┬───┘
│ │ │ │ │ │ │ │ │ │ │ │
│ ┌─────────────────┘ │ │ │ │ │ │ │ │ │ └──────────┐
│ │ ┌───────────────────────┘ │ │ │ │ │ │ │ │ │
│ │ │ ┌─────────────────┘ │ │ │ │ │ └─────────────┐ │ │
│ │ │ │ │ │ │ │ └──────┐ │ │ │
│ │ │ ┌───────────┼───────────────────────┘ │ │ │ │ │ │ │
│ │ │ │ │ ┌────────────────┘ │ │ │ │ │ │
│ │ │ │ │ │ ┌───────────────┘ │ │ │ │ │
│ │ │ │ │ │ │ │ │ │ │ │
v v v v v v v v v v v v
┌─────────────────┐ ┌─────────────────┐ ┌───────────┐ ┌───────────┐ ┌───────────┐ ┌─────────────────┐ ┌───────────┐ ┌───────────┐ ┌─────────────────┐ ┌────────────┐
│Literal[Int32]: 4│ │Literal[Int32]: 2│ │Variable[a]│ │Variable[c]│ │Variable[a]│ │Literal[Int32]: 1│ │Variable[b]│ │Variable[b]│ │Literal[Int32]: 1│ │Arguments(0)│
└─────────────────┘ └─────────────────┘ └───────────┘ └───────────┘ └───────────┘ └─────────────────┘ └───────────┘ └───────────┘ └─────────────────┘ └────────────┘
In [6]:
Copied!
# Create PNG representation
module
# Create PNG representation
module
Out[6]:
We can also get the PNG representation of parts of the AST, such as basic_op and the variable a:
In [7]:
Copied!
# Create PNG representation
basic_op
# Create PNG representation
basic_op
Out[7]:
In [8]:
Copied!
# Create PNG representation
a
# Create PNG representation
a
Out[8]:
Custom shapes¶
It is also possible to use custom shapes for the output using the function viz.visualize. The Default shape is box, but diamond, ellipse, and circle are also avaiable options.
In [9]:
Copied!
# Import visualization module
from astx import viz
# Create PNG representation with diamond shape
viz.visualize(a.get_struct(), shape="diamond")
# Import visualization module
from astx import viz
# Create PNG representation with diamond shape
viz.visualize(a.get_struct(), shape="diamond")
In [10]:
Copied!
# Create PNG representation with circle shape
viz.visualize(a.get_struct(), shape="circle")
# Create PNG representation with circle shape
viz.visualize(a.get_struct(), shape="circle")
In [11]:
Copied!
# Create PNG representation with ellipse shape
viz.visualize(a.get_struct(), shape="ellipse")
# Create PNG representation with ellipse shape
viz.visualize(a.get_struct(), shape="ellipse")
