This package offers both a portable {py3} module and a command-line
tool.
-WARNING: This version of Normand is 0.1, meaning both the Normand
+WARNING: This version of Normand is 0.6, meaning both the Normand
language and the module/CLI interface aren't stable.
ifdef::env-github[]
The value of a variable assignment is the evaluation of a valid {py3}
expression which may include label and variable names.
-Value encoding with a specific length (8{nbsp}bits to 64{nbsp}bits) and byte order::
+Fixed-length number with a given length (8{nbsp}bits to 64{nbsp}bits) and byte order::
+
Input:
+
{strength = 4}
{be} 67 <lbl> 44 $178 {(end - lbl) * 8 + strength : 16} $99 <end>
{le} {-1993 : 32}
+{-3.141593 : 64}
----
+
Output:
+
----
-67 44 b2 00 2c 63 37 f8 ff ff
+67 44 b2 00 2c 63 37 f8 ff ff 7f bd c2 82 fb 21
+09 c0
----
+
-The encoded value is the evaluation of a valid {py3} expression which
+The encoded number is the evaluation of a valid {py3} expression which
+may include label and variable names.
+
+https://en.wikipedia.org/wiki/LEB128[LEB128] integer::
++
+Input:
++
+----
+aa bb cc {-1993 : sleb128} <meow> dd ee ff
+{meow * 199 : uleb128}
+----
++
+Output:
++
+----
+aa bb cc b7 70 dd ee ff e3 07
+----
++
+The encoded integer is the evaluation of a valid {py3} expression which
may include label and variable names.
Repetition::
Input:
+
----
-aa bb * 5 cc "yeah\0" * 8
+aa bb * 5 cc <zoom> "yeah\0" * {zoom * 3}
----
+
Output:
+
----
-aa bb bb bb bb bb cc 79 65 61 68 00 79 65 61 68 ┆ •••••••yeah.yeah
+aa bb bb bb bb bb cc 79 65 61 68 00 79 65 61 68 ┆ •••••••yeah•yeah
+00 79 65 61 68 00 79 65 61 68 00 79 65 61 68 00 ┆ •yeah•yeah•yeah•
+79 65 61 68 00 79 65 61 68 00 79 65 61 68 00 79 ┆ yeah•yeah•yeah•y
+65 61 68 00 79 65 61 68 00 79 65 61 68 00 79 65 ┆ eah•yeah•yeah•ye
+61 68 00 79 65 61 68 00 79 65 61 68 00 79 65 61 ┆ ah•yeah•yeah•yea
+68 00 79 65 61 68 00 79 65 61 68 00 79 65 61 68 ┆ h•yeah•yeah•yeah
00 79 65 61 68 00 79 65 61 68 00 79 65 61 68 00 ┆ •yeah•yeah•yeah•
-79 65 61 68 00 79 65 61 68 00 79 65 61 68 00 ┆ yeah•yeah•yeah•
----
----
+
----
-/tmp/meow.normand:24:19 - Unknown variable/label name `meow` in expression `(meow - 45) // 8`.
+/tmp/meow.normand:24:19 - Illegal (unknown or unreachable) variable/label name `meow` in expression `(meow - 45) // 8`; the legal names are {`mix`, `zoom`}.
----
+
----
[NOTE]
====
Normand has a single module file, `normand.py`, which you can copy as is
-to your project to use it (both the <<python-3-api,`normand.parse()`>>
+to your project to use it (both the <<python3-api,`normand.parse()`>>
function and the <<command-line-tool,command-line tool>>).
`normand.py` has _no external dependencies_, but if you're using
[%header%autowidth]
|===
-|State variable |Description |Initial value: <<python-3-api,{py3} API>> |Initial value: <<command-line-tool,CLI>>
+|State variable |Description |Initial value: <<python3-api,{py3} API>> |Initial value: <<command-line-tool,CLI>>
|[[cur-offset]] Current offset
|
-The current offset has an effect on the value of
-<<label,labels>> and of the special `ICITTE` name in <<value,value>> and
+The current offset has an effect on the value of <<label,labels>> and of
+the special `ICITTE` name in <<fixed-length-number,fixed-length
+number>>, <<leb-128-integer,LEB128 integer>>, and
<<variable-assignment,variable assignment>> expression evaluation.
Each generated byte increments the current offset.
|[[cur-bo]] Current byte order
|
-The current byte order has an effect on the encoding of <<value,values>>.
+The current byte order has an effect on the encoding of
+<<fixed-length-number,fixed-length numbers>>.
A <<current-byte-order-setting,current byte order setting>> may change
the current byte order.
* A <<current-byte-order-setting,current byte order setting>> (big or
little endian).
-* A <<value,{py3} expression to be evaluated>> as an unsigned or signed
- integer to be encoded on one or more bytes using the current byte
- order.
+* A <<fixed-length-number,fixed-length number>> (integer or
+ floating point) using the <<cur-bo,current byte order>> and of which
+ the value is the result of a {py3} expression.
+
+* An <<leb128-integer,LEB128 integer>> of which the value is the result
+ of a {py3} expression.
* A <<current-offset-setting,current offset setting>>.
* A <<group,group>>, that is, a scoped sequence of items.
Moreover, you can <<repetition,repeat>> any item above, except an offset
-or a label, a given number of times. This is called a repetition.
+or a label, a given fixed or variable number of times. This is called a
+repetition.
A Normand comment may exist:
* Between the nibbles of a constant hexadecimal byte.
* Between the bits of a constant binary byte.
* Between the last item and the ``pass:[*]`` character of a repetition,
- and between that ``pass:[*]`` character and the following number.
+ and between that ``pass:[*]`` character and the following number
+ or expression.
A comment is anything between two ``pass:[#]`` characters on the same
line, or from ``pass:[#]`` until the end of the line. Whitespaces and
``pass:[{be}]``:: Set the current byte order to big endian.
``pass:[{le}]``:: Set the current byte order to little endian.
-=== Value
+=== Fixed-length number
-A _value_ represents a fixed number of bytes encoding an unsigned or
-signed integer which is the result of evaluating a {py3} expression
-using the <<cur-bo,current byte order>>.
+A _fixed-length number_ represents a fixed number of bytes encoding
+either:
-For a value at some source location{nbsp}__**L**__, its {py3} expression
-may contain the name of any accessible <<label,label>>, including the
-name of a label defined after{nbsp}__**L**__, as well as the name of any
-<<variable-assignment,variable>> known at{nbsp}__**L**__.
-
-An accessible label is either:
+* An unsigned or signed integer (two's complement).
++
+The available lengths are 8, 16, 24, 32, 40, 48, 56, and 64.
-* Outside of the current <<group,group>>.
-* Within the same immediate group (not within a nested group).
+* A floating point number
+ ([IEEE{nbsp}754-2008[https://standards.ieee.org/standard/754-2008.html]).
++
+The available length are 32 (_binary32_) and 64 (_binary64_).
-In the {py3} expression of a value, the value of the special name
-`ICITTE` is the <<cur-offset,current offset>> (before encoding the
-value).
+The value is the result of evaluating a {py3} expression using the
+<<cur-bo,current byte order>>.
-A value is:
+A fixed-length number is:
. The ``pass:[{]`` prefix.
. A valid {py3} expression.
++
+For a fixed-length number at some source location{nbsp}__**L**__, this
+expression may contain the name of any accessible <<label,label>> (not
+within a nested group), including the name of a label defined
+after{nbsp}__**L**__, as well as the name of any
+<<variable-assignment,variable>> known at{nbsp}__**L**__.
++
+The value of the special name `ICITTE` (`int` type) in this expression
+is the <<cur-offset,current offset>> (before encoding the number).
. The `:` character.
-. An encoding length in bits amongst `8`, `16`, `24`, `32`, `40`,
- `48`, `56`, and `64`.
+. An encoding length in bits amongst:
++
+--
+The expression evaluates to an `int` value::
+ `8`, `16`, `24`, `32`, `40`, `48`, `56`, and `64`.
+
+The expression evaluates to a `float` value::
+ `32` and `64`.
+--
. The `}` suffix.
----
====
+====
+Input:
+
+----
+{le}
+{2 * 0.0529 : 32}
+----
+
+Output:
+
+----
+ac ad d8 3d
+----
+====
+
+=== LEB128 integer
+
+An _LEB128 integer_ represents a variable number of bytes encoding an
+unsigned or signed integer which is the result of evaluating a {py3}
+expression following the https://en.wikipedia.org/wiki/LEB128[LEB128]
+format.
+
+An LEB128 integer is:
+
+. The ``pass:[{]`` prefix.
+
+. A valid {py3} expression.
++
+For an LEB128 integer at some source location{nbsp}__**L**__, this
+expression may contain:
++
+--
+* The name of any <<label,label>> defined before{nbsp}__**L**__.
+* The name of any <<variable-assignment,variable>> known at{nbsp}__**L**__
+ which doesn't, directly or indirectly, refer to a label
+ defined after{nbsp}__**L**__.
+--
++
+The value of the special name `ICITTE` (`int` type) in this expression
+is the <<cur-offset,current offset>> (before encoding the integer).
+
+. The `:` character.
+
+. One of:
++
+--
+[horizontal]
+`uleb128`:: Use the unsigned LEB128 format.
+`sleb128`:: Use the signed LEB128 format.
+--
+
+. The `}` suffix.
+
+====
+Input:
+
+----
+{624485 : uleb128}
+----
+
+Output:
+
+----
+e5 8e 26
+----
+====
+
+====
+Input:
+
+----
+aa bb cc dd
+<meow>
+ee ff
+{-981238311 + (meow * -23) : sleb128}
+"hello"
+----
+
+Output:
+
+----
+aa bb cc dd ee ff fd fa 8d ac 7c 68 65 6c 6c 6f ┆ ••••••••••|hello
+----
+====
+
=== Current offset setting
This special item sets the <<cur-offset,_current offset_>>.
All the labels of a whole Normand input must have unique names.
-A label may not share the name of a <<variable-assignment,variable>>
+A label must not share the name of a <<variable-assignment,variable>>
name.
-A label name may not be `ICITTE` (see <<value>> and
-<<variable-assignment>> to learn more).
-
A label is:
. The `<` prefix.
-. A valid {py3} name which is not `ICITTE`.
+. A valid {py3} name which is not `ICITTE` (see
+ <<fixed-length-number>>, <<leb128-integer>>, and
+ <<variable-assignment>> to learn more).
. The `>` suffix.
A _variable assignment_ associates a name to the integral result of an
evaluated {py3} expression.
-For a variable assignment at some source location{nbsp}__**L**__, its
-{py3} expression may contain the name of any accessible <<label,label>>,
-including the name of a label defined after{nbsp}__**L**__, as well as
-the name of any variable known at{nbsp}__**L**__.
-
-An accessible label is either:
-
-* Outside of the current <<group,group>>.
-* Within the same immediate group (not within a nested group).
-
-A variable name may not be `ICITTE`.
-
-In the {py3} expression of a variable assignment, the special name
-`ICITTE` is the <<cur-offset,current offset>>.
-
-A variable is:
+A variable assignment is:
. The ``pass:[{]`` prefix.
-. A valid {py3} name which is not `ICITTE`.
+. A valid {py3} name which is not `ICITTE` (see
+ <<fixed-length-number>>, <<leb128-integer>>, and
+ <<variable-assignment>> to learn more).
. The `=` character.
. A valid {py3} expression.
++
+For a variable assignment at some source location{nbsp}__**L**__, this
+expression may contain the name of any accessible <<label,label>> (not
+within a nested group), including the name of a label defined
+after{nbsp}__**L**__, as well as the name of any
+<<variable-assignment,variable>> known at{nbsp}__**L**__.
++
+The value of the special name `ICITTE` (`int` type) in this expression
+is the <<cur-offset,current offset>>.
. The `}` suffix.
. The ``pass:[*]`` character.
-. A positive integer (hexadecimal starting with `0x` or `0X` accepted)
- which is the number of times to repeat the previous item.
+. One of:
+
+** A positive integer (hexadecimal starting with `0x` or `0X` accepted)
+ which is the number of times to repeat the previous item.
+
+** The ``pass:[{]`` prefix, a valid {py3} expression, and the
+ ``pass:[}]`` suffix.
++
+For a repetition at some source location{nbsp}__**L**__, this expression
+may contain:
++
+--
+* The name of any <<label,label>> defined before{nbsp}__**L**__ and
+ which isn't part of its repeated item.
+* The name of any <<variable-assignment,variable>> known
+ at{nbsp}__**L**__, which isn't part of its repeated item, and which
+ doesn't, directly or indirectly, refer to a label defined
+ after{nbsp}__**L**__.
+--
++
+This expression must not contain the special name `ICITTE`.
====
Input:
----
====
+====
+Input:
+
+----
+{times = 1}
+aa bb cc dd
+(
+ <here>
+ (ee ff) * {here + 1}
+ 11 22 33 * {times}
+ {times = times + 1}
+) * 3
+"coucou!"
+----
+
+Output:
+
+----
+aa bb cc dd ee ff ee ff ee ff ee ff ee ff 11 22 ┆ •••••••••••••••"
+33 ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ 3•••••••••••••••
+ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
+ff ee ff ee ff 11 22 33 33 ee ff ee ff ee ff ee ┆ ••••••"33•••••••
+ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
+ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
+ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
+ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
+ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
+ff ee ff ee ff ee ff ee ff ee ff ee ff ee ff ee ┆ ••••••••••••••••
+ff ee ff ee ff ee ff ee ff ee ff ee ff 11 22 33 ┆ ••••••••••••••"3
+33 33 63 6f 75 63 6f 75 21 ┆ 33coucou!
+----
+====
+
+====
+This example shows how to use a repetition as a conditional section
+depending on some predefined variable.
+
+Input:
+
+----
+aa bb cc dd
+(ee ff "meow mix" 00) * {cond}
+{be} {-1993:16}
+----
+
+Output (`cond` is 0):
+
+----
+aa bb cc dd f8 37
+----
+
+Output (`cond` is 1):
+
+----
+aa bb cc dd ee ff 6d 65 6f 77 20 6d 69 78 00 f8 ┆ ••••••meow mix••
+37 ┆ 7
+----
+====
+
== Command-line tool
If you <<install-normand,installed>> the `normand` package, then you
LE = ...
-VarsT = typing.Dict[str, int]
-
-
class TextLoc:
# Line number.
@property
...
+SymbolsT = typing.Dict[str, int]
+
+
class ParseResult:
# Generated data.
@property
# Updated variable values.
@property
- def variables(self) -> VarsT:
+ def variables(self) -> SymbolsT:
...
# Updated main group label values.
@property
- def labels(self) -> VarsT:
+ def labels(self) -> SymbolsT:
...
# Final offset.
# Final byte order.
@property
- def byte_order(self) -> typing.Optional[int]:
+ def byte_order(self) -> typing.Optional[ByteOrder]:
...
+
def parse(normand: str,
- init_variables: typing.Optional[VarsT] = None,
- init_labels: typing.Optional[VarsT] = None,
+ init_variables: typing.Optional[SymbolsT] = None,
+ init_labels: typing.Optional[SymbolsT] = None,
init_offset: int = 0,
init_byte_order: typing.Optional[ByteOrder] = None) -> ParseResult:
...
The `parse()` function raises a `ParseError` instance should it fail to
parse the `normand` string for any reason.
+
+== Development
+
+Normand is a https://python-poetry.org/[Poetry] project.
+
+To develop it, install it through Poetry and enter the virtual
+environment:
+
+----
+$ poetry install
+$ poetry shell
+$ normand <<< '"lol" * 10 0a'
+----
+
+`normand.py` is processed by:
+
+* https://microsoft.github.io/pyright/[Pyright]
+* https://github.com/psf/black[Black]
+* https://pycqa.github.io/isort/[isort]
+
+=== Testing
+
+Use https://docs.pytest.org/[pytest] to test Normand once the package is
+part of your virtual environment, for example:
+
+----
+$ poetry install
+$ poetry run pip3 install pytest
+$ poetry run pytest
+----
+
+The `pytest` project is currently not a development dependency in
+`pyproject.toml` due to backward compatibiliy issues with
+Python{nbsp}3.4.
+
+In the `tests` directory, each `*.nt` file is a test. The file name
+prefix indicates what it's meant to test:
+
+`pass-`::
+ Everything above the `---` line is the valid Normand input
+ to test.
++
+Everything below the `---` line is the expected data
+(whitespace-separated hexadecimal bytes).
+
+`fail-`::
+ Everything above the `---` line is the invalid Normand input
+ to test.
++
+Everything below the `---` line is the expected error message having
+this form:
++
+----
+LINE:COL - MESSAGE
+----
+
+=== Contributing
+
+Normand uses https://review.lttng.org/admin/repos/normand,general[Gerrit]
+for code review.
+
+To report a bug, https://github.com/efficios/normand/issues/new[create a
+GitHub issue].
projects / normand.git / blobdiff