160f02e4d0
This avoids double initialization when decoding nested submessages. Fixes an issue with submessage arrays that was present in previous version of this patch. Update issue 28 Status: FixedInGit
456 lines
22 KiB
ReStructuredText
456 lines
22 KiB
ReStructuredText
=====================
|
|
Nanopb: API reference
|
|
=====================
|
|
|
|
.. include :: menu.rst
|
|
|
|
.. contents ::
|
|
|
|
Compilation options
|
|
===================
|
|
The following options can be specified using -D switch given to the C compiler:
|
|
|
|
============================ ================================================================================================
|
|
__BIG_ENDIAN__ Set this if your platform stores integers and floats in big-endian format.
|
|
Mixed-endian systems (different layout for ints and floats) are currently not supported.
|
|
NANOPB_INTERNALS Set this to expose the field encoder functions that are hidden since nanopb-0.1.3.
|
|
PB_MAX_REQUIRED_FIELDS Maximum number of required fields to check for presence. Default value is 64. Increases stack
|
|
usage 1 byte per every 8 fields. Compiler warning will tell if you need this.
|
|
PB_FIELD_16BIT Add support for tag numbers > 255 and fields larger than 255 bytes or 255 array entries.
|
|
Increases code size 3 bytes per each field. Compiler error will tell if you need this.
|
|
PB_FIELD_32BIT Add support for tag numbers > 65535 and fields larger than 65535 bytes or 65535 array entries.
|
|
Increases code size 9 bytes per each field. Compiler error will tell if you need this.
|
|
PB_NO_ERRMSG Disables the support for error messages; only error information is the true/false return value.
|
|
Decreases the code size by a few hundred bytes.
|
|
============================ ================================================================================================
|
|
|
|
The PB_MAX_REQUIRED_FIELDS, PB_FIELD_16BIT and PB_FIELD_32BIT settings allow raising some datatype limits to suit larger messages.
|
|
Their need is recognized automatically by C-preprocessor #if-directives in the generated .pb.h files. The default setting is to use
|
|
the smallest datatypes (least resources used).
|
|
|
|
pb.h
|
|
====
|
|
|
|
pb_type_t
|
|
---------
|
|
Defines the encoder/decoder behaviour that should be used for a field. ::
|
|
|
|
typedef enum { ... } pb_type_t;
|
|
|
|
The low-order byte of the enumeration values defines the function that can be used for encoding and decoding the field data:
|
|
|
|
==================== ===== ================================================
|
|
LTYPE identifier Value Storage format
|
|
==================== ===== ================================================
|
|
PB_LTYPE_VARINT 0x00 Integer.
|
|
PB_LTYPE_SVARINT 0x01 Integer, zigzag encoded.
|
|
PB_LTYPE_FIXED 0x02 Integer or floating point.
|
|
PB_LTYPE_BYTES 0x03 Structure with *size_t* field and byte array.
|
|
PB_LTYPE_STRING 0x04 Null-terminated string.
|
|
PB_LTYPE_SUBMESSAGE 0x05 Submessage structure.
|
|
==================== ===== ================================================
|
|
|
|
The high-order byte defines whether the field is required, optional, repeated or callback:
|
|
|
|
==================== ===== ================================================
|
|
HTYPE identifier Value Field handling
|
|
==================== ===== ================================================
|
|
PB_HTYPE_REQUIRED 0x00 Verify that field exists in decoded message.
|
|
PB_HTYPE_OPTIONAL 0x10 Use separate *has_<field>* boolean to specify
|
|
whether the field is present.
|
|
PB_HTYPE_ARRAY 0x20 A repeated field with preallocated array.
|
|
Separate *<field>_count* for number of items.
|
|
PB_HTYPE_CALLBACK 0x30 A field with dynamic storage size, data is
|
|
actually a pointer to a structure containing a
|
|
callback function.
|
|
==================== ===== ================================================
|
|
|
|
pb_field_t
|
|
----------
|
|
Describes a single structure field with memory position in relation to others. The descriptions are usually autogenerated. ::
|
|
|
|
typedef struct _pb_field_t pb_field_t;
|
|
struct _pb_field_t {
|
|
uint8_t tag;
|
|
pb_type_t type;
|
|
uint8_t data_offset;
|
|
int8_t size_offset;
|
|
uint8_t data_size;
|
|
uint8_t array_size;
|
|
const void *ptr;
|
|
} pb_packed;
|
|
|
|
:tag: Tag number of the field or 0 to terminate a list of fields.
|
|
:type: LTYPE and HTYPE of the field.
|
|
:data_offset: Offset of field data, relative to the end of the previous field.
|
|
:size_offset: Offset of *bool* flag for optional fields or *size_t* count for arrays, relative to field data.
|
|
:data_size: Size of a single data entry, in bytes. For PB_LTYPE_BYTES, the size of the byte array inside the containing structure. For PB_HTYPE_CALLBACK, size of the C data type if known.
|
|
:array_size: Maximum number of entries in an array, if it is an array type.
|
|
:ptr: Pointer to default value for optional fields, or to submessage description for PB_LTYPE_SUBMESSAGE.
|
|
|
|
The *uint8_t* datatypes limit the maximum size of a single item to 255 bytes and arrays to 255 items. Compiler will give error if the values are too large. The types can be changed to larger ones by defining *PB_FIELD_16BIT*.
|
|
|
|
pb_bytes_array_t
|
|
----------------
|
|
An byte array with a field for storing the length::
|
|
|
|
typedef struct {
|
|
size_t size;
|
|
uint8_t bytes[1];
|
|
} pb_bytes_array_t;
|
|
|
|
In an actual array, the length of *bytes* may be different.
|
|
|
|
pb_callback_t
|
|
-------------
|
|
Part of a message structure, for fields with type PB_HTYPE_CALLBACK::
|
|
|
|
typedef struct _pb_callback_t pb_callback_t;
|
|
struct _pb_callback_t {
|
|
union {
|
|
bool (*decode)(pb_istream_t *stream, const pb_field_t *field, void *arg);
|
|
bool (*encode)(pb_ostream_t *stream, const pb_field_t *field, const void *arg);
|
|
} funcs;
|
|
|
|
void *arg;
|
|
};
|
|
|
|
The *arg* is passed to the callback when calling. It can be used to store any information that the callback might need.
|
|
|
|
When calling `pb_encode`_, *funcs.encode* is used, and similarly when calling `pb_decode`_, *funcs.decode* is used. The function pointers are stored in the same memory location but are of incompatible types. You can set the function pointer to NULL to skip the field.
|
|
|
|
pb_wire_type_t
|
|
--------------
|
|
Protocol Buffers wire types. These are used with `pb_encode_tag`_. ::
|
|
|
|
typedef enum {
|
|
PB_WT_VARINT = 0,
|
|
PB_WT_64BIT = 1,
|
|
PB_WT_STRING = 2,
|
|
PB_WT_32BIT = 5
|
|
} pb_wire_type_t;
|
|
|
|
pb_encode.h
|
|
===========
|
|
|
|
pb_ostream_from_buffer
|
|
----------------------
|
|
Constructs an output stream for writing into a memory buffer. This is just a helper function, it doesn't do anything you couldn't do yourself in a callback function. It uses an internal callback that stores the pointer in stream *state* field. ::
|
|
|
|
pb_ostream_t pb_ostream_from_buffer(uint8_t *buf, size_t bufsize);
|
|
|
|
:buf: Memory buffer to write into.
|
|
:bufsize: Maximum number of bytes to write.
|
|
:returns: An output stream.
|
|
|
|
After writing, you can check *stream.bytes_written* to find out how much valid data there is in the buffer.
|
|
|
|
pb_write
|
|
--------
|
|
Writes data to an output stream. Always use this function, instead of trying to call stream callback manually. ::
|
|
|
|
bool pb_write(pb_ostream_t *stream, const uint8_t *buf, size_t count);
|
|
|
|
:stream: Output stream to write to.
|
|
:buf: Pointer to buffer with the data to be written.
|
|
:count: Number of bytes to write.
|
|
:returns: True on success, false if maximum length is exceeded or an IO error happens.
|
|
|
|
If an error happens, *bytes_written* is not incremented. Depending on the callback used, calling pb_write again after it has failed once may be dangerous. Nanopb itself never does this, instead it returns the error to user application. The builtin pb_ostream_from_buffer is safe to call again after failed write.
|
|
|
|
pb_encode
|
|
---------
|
|
Encodes the contents of a structure as a protocol buffers message and writes it to output stream. ::
|
|
|
|
bool pb_encode(pb_ostream_t *stream, const pb_field_t fields[], const void *src_struct);
|
|
|
|
:stream: Output stream to write to.
|
|
:fields: A field description array, usually autogenerated.
|
|
:src_struct: Pointer to the data that will be serialized.
|
|
:returns: True on success, false on IO error, on detectable errors in field description, or if a field encoder returns false.
|
|
|
|
Normally pb_encode simply walks through the fields description array and serializes each field in turn. However, submessages must be serialized twice: first to calculate their size and then to actually write them to output. This causes some constraints for callback fields, which must return the same data on every call.
|
|
|
|
.. sidebar:: Encoding fields manually
|
|
|
|
The functions with names *pb_encode_\** are used when dealing with callback fields. The typical reason for using callbacks is to have an array of unlimited size. In that case, `pb_encode`_ will call your callback function, which in turn will call *pb_encode_\** functions repeatedly to write out values.
|
|
|
|
The tag of a field must be encoded separately with `pb_encode_tag_for_field`_. After that, you can call exactly one of the content-writing functions to encode the payload of the field. For repeated fields, you can repeat this process multiple times.
|
|
|
|
Writing packed arrays is a little bit more involved: you need to use `pb_encode_tag` and specify `PB_WT_STRING` as the wire type. Then you need to know exactly how much data you are going to write, and use `pb_encode_varint`_ to write out the number of bytes before writing the actual data. Substreams can be used to determine the number of bytes beforehand; see `pb_encode_submessage`_ source code for an example.
|
|
|
|
pb_encode_tag
|
|
-------------
|
|
Starts a field in the Protocol Buffers binary format: encodes the field number and the wire type of the data. ::
|
|
|
|
bool pb_encode_tag(pb_ostream_t *stream, pb_wire_type_t wiretype, int field_number);
|
|
|
|
:stream: Output stream to write to. 1-5 bytes will be written.
|
|
:wiretype: PB_WT_VARINT, PB_WT_64BIT, PB_WT_STRING or PB_WT_32BIT
|
|
:field_number: Identifier for the field, defined in the .proto file. You can get it from field->tag.
|
|
:returns: True on success, false on IO error.
|
|
|
|
pb_encode_tag_for_field
|
|
-----------------------
|
|
Same as `pb_encode_tag`_, except takes the parameters from a *pb_field_t* structure. ::
|
|
|
|
bool pb_encode_tag_for_field(pb_ostream_t *stream, const pb_field_t *field);
|
|
|
|
:stream: Output stream to write to. 1-5 bytes will be written.
|
|
:field: Field description structure. Usually autogenerated.
|
|
:returns: True on success, false on IO error or unknown field type.
|
|
|
|
This function only considers the LTYPE of the field. You can use it from your field callbacks, because the source generator writes correct LTYPE also for callback type fields.
|
|
|
|
Wire type mapping is as follows:
|
|
|
|
========================= ============
|
|
LTYPEs Wire type
|
|
========================= ============
|
|
VARINT, SVARINT PB_WT_VARINT
|
|
FIXED64 PB_WT_64BIT
|
|
STRING, BYTES, SUBMESSAGE PB_WT_STRING
|
|
FIXED32 PB_WT_32BIT
|
|
========================= ============
|
|
|
|
pb_encode_varint
|
|
----------------
|
|
Encodes a signed or unsigned integer in the varint_ format. Works for fields of type `bool`, `enum`, `int32`, `int64`, `uint32` and `uint64`::
|
|
|
|
bool pb_encode_varint(pb_ostream_t *stream, uint64_t value);
|
|
|
|
:stream: Output stream to write to. 1-10 bytes will be written.
|
|
:value: Value to encode. Just cast e.g. int32_t directly to uint64_t.
|
|
:returns: True on success, false on IO error.
|
|
|
|
.. _varint: http://code.google.com/apis/protocolbuffers/docs/encoding.html#varints
|
|
|
|
pb_encode_svarint
|
|
-----------------
|
|
Encodes a signed integer in the 'zig-zagged' format. Works for fields of type `sint32` and `sint64`::
|
|
|
|
bool pb_encode_svarint(pb_ostream_t *stream, int64_t value);
|
|
|
|
(parameters are the same as for `pb_encode_varint`_
|
|
|
|
pb_encode_string
|
|
----------------
|
|
Writes the length of a string as varint and then contents of the string. Works for fields of type `bytes` and `string`::
|
|
|
|
bool pb_encode_string(pb_ostream_t *stream, const uint8_t *buffer, size_t size);
|
|
|
|
:stream: Output stream to write to.
|
|
:buffer: Pointer to string data.
|
|
:size: Number of bytes in the string. Pass `strlen(s)` for strings.
|
|
:returns: True on success, false on IO error.
|
|
|
|
pb_encode_fixed32
|
|
-----------------
|
|
Writes 4 bytes to stream and swaps bytes on big-endian architectures. Works for fields of type `fixed32`, `sfixed32` and `float`::
|
|
|
|
bool pb_encode_fixed32(pb_ostream_t *stream, const void *value);
|
|
|
|
:stream: Output stream to write to.
|
|
:value: Pointer to a 4-bytes large C variable, for example `uint32_t foo;`.
|
|
:returns: True on success, false on IO error.
|
|
|
|
pb_encode_fixed64
|
|
-----------------
|
|
Writes 8 bytes to stream and swaps bytes on big-endian architecture. Works for fields of type `fixed64`, `sfixed64` and `double`::
|
|
|
|
bool pb_encode_fixed64(pb_ostream_t *stream, const void *value);
|
|
|
|
:stream: Output stream to write to.
|
|
:value: Pointer to a 8-bytes large C variable, for example `uint64_t foo;`.
|
|
:returns: True on success, false on IO error.
|
|
|
|
pb_encode_submessage
|
|
--------------------
|
|
Encodes a submessage field, including the size header for it. Works for fields of any message type::
|
|
|
|
bool pb_encode_submessage(pb_ostream_t *stream, const pb_field_t fields[], const void *src_struct);
|
|
|
|
:stream: Output stream to write to.
|
|
:fields: Pointer to the autogenerated field description array for the submessage type, e.g. `MyMessage_fields`.
|
|
:src: Pointer to the structure where submessage data is.
|
|
:returns: True on success, false on IO errors, pb_encode errors or if submessage size changes between calls.
|
|
|
|
In Protocol Buffers format, the submessage size must be written before the submessage contents. Therefore, this function has to encode the submessage twice in order to know the size beforehand.
|
|
|
|
If the submessage contains callback fields, the callback function might misbehave and write out a different amount of data on the second call. This situation is recognized and *false* is returned, but garbage will be written to the output before the problem is detected.
|
|
|
|
pb_decode.h
|
|
===========
|
|
|
|
pb_istream_from_buffer
|
|
----------------------
|
|
Helper function for creating an input stream that reads data from a memory buffer. ::
|
|
|
|
pb_istream_t pb_istream_from_buffer(uint8_t *buf, size_t bufsize);
|
|
|
|
:buf: Pointer to byte array to read from.
|
|
:bufsize: Size of the byte array.
|
|
:returns: An input stream ready to use.
|
|
|
|
pb_read
|
|
-------
|
|
Read data from input stream. Always use this function, don't try to call the stream callback directly. ::
|
|
|
|
bool pb_read(pb_istream_t *stream, uint8_t *buf, size_t count);
|
|
|
|
:stream: Input stream to read from.
|
|
:buf: Buffer to store the data to, or NULL to just read data without storing it anywhere.
|
|
:count: Number of bytes to read.
|
|
:returns: True on success, false if *stream->bytes_left* is less than *count* or if an IO error occurs.
|
|
|
|
End of file is signalled by *stream->bytes_left* being zero after pb_read returns false.
|
|
|
|
pb_decode
|
|
---------
|
|
Read and decode all fields of a structure. Reads until EOF on input stream. ::
|
|
|
|
bool pb_decode(pb_istream_t *stream, const pb_field_t fields[], void *dest_struct);
|
|
|
|
:stream: Input stream to read from.
|
|
:fields: A field description array. Usually autogenerated.
|
|
:dest_struct: Pointer to structure where data will be stored.
|
|
:returns: True on success, false on IO error, on detectable errors in field description, if a field encoder returns false or if a required field is missing.
|
|
|
|
In Protocol Buffers binary format, EOF is only allowed between fields. If it happens anywhere else, pb_decode will return *false*. If pb_decode returns false, you cannot trust any of the data in the structure.
|
|
|
|
In addition to EOF, the pb_decode implementation supports terminating a message with a 0 byte. This is compatible with the official Protocol Buffers because 0 is never a valid field tag.
|
|
|
|
For optional fields, this function applies the default value and sets *has_<field>* to false if the field is not present.
|
|
|
|
pb_decode_noinit
|
|
----------------
|
|
Same as `pb_decode`_, except does not apply the default values to fields. ::
|
|
|
|
bool pb_decode_noinit(pb_istream_t *stream, const pb_field_t fields[], void *dest_struct);
|
|
|
|
(parameters are the same as for `pb_decode`_.)
|
|
|
|
The destination structure should be filled with zeros before calling this function. Doing a *memset* manually can be slightly faster than using `pb_decode`_ if you don't need any default values.
|
|
|
|
pb_skip_varint
|
|
--------------
|
|
Skip a varint_ encoded integer without decoding it. ::
|
|
|
|
bool pb_skip_varint(pb_istream_t *stream);
|
|
|
|
:stream: Input stream to read from. Will read 1 byte at a time until the MSB is clear.
|
|
:returns: True on success, false on IO error.
|
|
|
|
pb_skip_string
|
|
--------------
|
|
Skip a varint-length-prefixed string. This means skipping a value with wire type PB_WT_STRING. ::
|
|
|
|
bool pb_skip_string(pb_istream_t *stream);
|
|
|
|
:stream: Input stream to read from.
|
|
:returns: True on success, false on IO error or length exceeding uint32_t.
|
|
|
|
pb_decode_tag
|
|
-------------
|
|
Decode the tag that comes before field in the protobuf encoding::
|
|
|
|
bool pb_decode_tag(pb_istream_t *stream, pb_wire_type_t *wire_type, int *tag, bool *eof);
|
|
|
|
:stream: Input stream to read from.
|
|
:wire_type: Pointer to variable where to store the wire type of the field.
|
|
:tag: Pointer to variable where to store the tag of the field.
|
|
:eof: Pointer to variable where to store end-of-file status.
|
|
:returns: True on success, false on error or EOF.
|
|
|
|
When the message (stream) ends, this function will return false and set *eof* to true. On other
|
|
errors, *eof* will be set to false.
|
|
|
|
pb_skip_field
|
|
-------------
|
|
Remove the data for a field from the stream, without actually decoding it::
|
|
|
|
bool pb_skip_field(pb_istream_t *stream, pb_wire_type_t wire_type);
|
|
|
|
:stream: Input stream to read from.
|
|
:wire_type: Type of field to skip.
|
|
:returns: True on success, false on IO error.
|
|
|
|
.. sidebar:: Decoding fields manually
|
|
|
|
The functions with names beginning with *pb_decode_* are used when dealing with callback fields. The typical reason for using callbacks is to have an array of unlimited size. In that case, `pb_decode`_ will call your callback function repeatedly, which can then store the values into e.g. filesystem in the order received in.
|
|
|
|
For decoding numeric (including enumerated and boolean) values, use `pb_decode_varint`_, `pb_decode_svarint`_, `pb_decode_fixed32`_ and `pb_decode_fixed64`_. They take a pointer to a 32- or 64-bit C variable, which you may then cast to smaller datatype for storage.
|
|
|
|
For decoding strings and bytes fields, the length has already been decoded. You can therefore check the total length in *stream->state* and read the data using `pb_read`_.
|
|
|
|
Finally, for decoding submessages in a callback, simply use `pb_decode`_ and pass it the *SubMessage_fields* descriptor array.
|
|
|
|
pb_decode_varint
|
|
----------------
|
|
Read and decode a varint_ encoded integer. ::
|
|
|
|
bool pb_decode_varint(pb_istream_t *stream, uint64_t *dest);
|
|
|
|
:stream: Input stream to read from. 1-10 bytes will be read.
|
|
:dest: Storage for the decoded integer. Value is undefined on error.
|
|
:returns: True on success, false if value exceeds uint64_t range or an IO error happens.
|
|
|
|
pb_decode_svarint
|
|
-----------------
|
|
Similar to `pb_decode_varint`_, except that it performs zigzag-decoding on the value. This corresponds to the Protocol Buffers *sint32* and *sint64* datatypes. ::
|
|
|
|
bool pb_decode_svarint(pb_istream_t *stream, int64_t *dest);
|
|
|
|
(parameters are the same as `pb_decode_varint`_)
|
|
|
|
pb_decode_fixed32
|
|
-----------------
|
|
Decode a *fixed32*, *sfixed32* or *float* value. ::
|
|
|
|
bool pb_decode_fixed32(pb_istream_t *stream, void *dest);
|
|
|
|
:stream: Input stream to read from. 4 bytes will be read.
|
|
:dest: Pointer to destination *int32_t*, *uint32_t* or *float*.
|
|
:returns: True on success, false on IO errors.
|
|
|
|
This function reads 4 bytes from the input stream.
|
|
On big endian architectures, it then reverses the order of the bytes.
|
|
Finally, it writes the bytes to *dest*.
|
|
|
|
pb_decode_fixed64
|
|
-----------------
|
|
Decode a *fixed64*, *sfixed64* or *double* value. ::
|
|
|
|
bool pb_dec_fixed(pb_istream_t *stream, const pb_field_t *field, void *dest);
|
|
|
|
:stream: Input stream to read from. 8 bytes will be read.
|
|
:field: Not used.
|
|
:dest: Pointer to destination *int64_t*, *uint64_t* or *double*.
|
|
:returns: True on success, false on IO errors.
|
|
|
|
Same as `pb_decode_fixed32`_, except this reads 8 bytes.
|
|
|
|
pb_make_string_substream
|
|
------------------------
|
|
Decode the length for a field with wire type *PB_WT_STRING* and create a substream for reading the data. ::
|
|
|
|
bool pb_make_string_substream(pb_istream_t *stream, pb_istream_t *substream);
|
|
|
|
:stream: Original input stream to read the length and data from.
|
|
:substream: New substream that has limited length. Filled in by the function.
|
|
:returns: True on success, false if reading the length fails.
|
|
|
|
This function uses `pb_decode_varint`_ to read an integer from the stream. This is interpreted as a number of bytes, and the substream is set up so that its `bytes_left` is initially the same as the length, and its callback function and state the same as the parent stream.
|
|
|
|
pb_close_string_substream
|
|
-------------------------
|
|
Close the substream created with `pb_make_string_substream`_. ::
|
|
|
|
void pb_close_string_substream(pb_istream_t *stream, pb_istream_t *substream);
|
|
|
|
:stream: Original input stream to read the length and data from.
|
|
:substream: Substream to close
|
|
|
|
This function copies back the state from the substream to the parent stream.
|
|
It must be called after done with the substream.
|