rik/nanopb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Support dynamic allocation for string, bytes and message fields.

Browse Source 
This is turned on by passing -p to nanopb_generator.py or setting the
(nanopb).pointer option for a .proto field.

git-svn-id: https://svn.kapsi.fi/jpa/nanopb-dev@1081 e3a754e5-d11d-0410-8d38-ebb782a927b9
This commit is contained in:
Michael Poole
2011-12-20 03:30:52 +00:00
committed by Petteri Aimonen
parent 8e5337e9ef
commit c66c6b43c4
14 changed files with 403 additions and 48 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
docs/concepts.rst
View File

@@ -142,6 +142,7 @@ Most Protocol Buffers datatypes have directly corresponding C datatypes, such as
1) Strings, bytes and repeated fields of any type map to callback functions by default.
2) If there is a special option *(nanopb).max_size* specified in the .proto file, string maps to null-terminated char array and bytes map to a structure containing a char array and a size field.
3) If there is a special option *(nanopb).max_count* specified on a repeated field, it maps to an array of whatever type is being repeated. Another field will be created for the actual number of entries stored.
4) The option *(nanopb).pointer* can override the default (false, unless the *-p* option is passed to *nanopb_generator.py*) behavior. If a string, byte, or submessage is generated as a pointer field, and it is repeated (with a maximum count), required, or optional, the members will be pointers rather than in-line data.
=============================================================================== =======================
field in .proto autogenerated in .h
@@ -156,10 +157,19 @@ required bytes data = 1 [(nanopb).max_size = 40];
| uint8_t bytes[40];
| } Person_data_t;
| Person_data_t data;
required string name = 1 [(nanopb).pointer = true]; char \*name;
required bytes data = 1 [(nanopb).pointer = true]; pb_bytes_t data;
required bytes data = 1 [(nanopb).pointer = true, (nanopb).max_count = 5]; | size_t data_count;
| pb_bytes_t data[5];
required Message msg = 1 [(nanopb).pointer = true]; Message \*msg;
=============================================================================== =======================
The maximum lengths are checked in runtime. If string/bytes/array exceeds the allocated length, *pb_decode* will return false.
If a pointer-type field is not received, the field will be marked as absent, but the pointer will not be modified. This helps reduce memory fragmentation and churn, but increases worst-case memory usage and means you must use the *Message_has(object, Field)* macro rather than testing for a null pointer. You can use the `pb_clean`_ function to release unused memory in these cases.
.. _`pb_clean`: reference.html#pb-clean
Field callbacks
===============
When a field has dynamic length, nanopb cannot statically allocate storage for it. Instead, it allows you to handle the field in whatever way you want, using a callback function.
@@ -279,6 +289,8 @@ For convenience, *pb_encode* only checks these bits for optional
fields. *pb_decode* sets the corresponding bit for every field it
decodes, whether the field is optional or not.
.. Should there be a section here on pointer fields?
Return values and error handling
================================

4
docs/index.rst
View File

@@ -15,6 +15,8 @@ Overall structure
For the runtime program, you always need *pb.h* for type declarations.
Depending on whether you want to encode, decode, or both, you also need *pb_encode.h/c* or *pb_decode.h/c*.
If your *.proto* file encodes submessages or other fields using pointers, you must compile *pb_decode.c* with a preprocessor macro named *MALLOC_HEADER* that is the name of a header with definitions (either as functions or macros) for *malloc()*, *realloc()* and *free()*. For a typical hosted configuration, this should be *<stdlib.h>*.
The high-level encoding and decoding functions take an array of *pb_field_t* structures, which describes the fields of a message structure. Usually you want these autogenerated from a *.proto* file. The tool script *nanopb_generator.py* accomplishes this.
.. image:: generator_flow.png
@@ -52,7 +54,7 @@ Features and limitations
#) Unknown fields are not preserved when decoding and re-encoding a message.
#) Reflection (runtime introspection) is not supported. E.g. you can't request a field by giving its name in a string.
#) Numeric arrays are always encoded as packed, even if not marked as packed in .proto. This causes incompatibility with decoders that do not support packed format.
#) Cyclic references between messages are not supported. They could be supported in callback-mode if there was an option in the generator to set the mode.
#) Limited support for cyclic references between messages. (The cycle must be broken by making one of the references a pointer or callback field, and objects that have circular references are not detected when encoding.)
Getting started
===============

32
docs/reference.rst
View File

@@ -28,7 +28,7 @@ PB_LTYPE_STRING 0x04 Null-terminated string.
PB_LTYPE_SUBMESSAGE 0x05 Submessage structure.
==================== ===== ================================================
The high-order nibble defines whether the field is required, optional, repeated or callback:
The high-order nibble defines whether the field is required, optional, repeated or callback, and whether it is a pointer:
==================== ===== ================================================
HTYPE identifier Value Field handling
@@ -41,6 +41,10 @@ PB_HTYPE_ARRAY 0x20 A repeated field with preallocated array.
PB_HTYPE_CALLBACK 0x30 A field with dynamic storage size, data is
actually a pointer to a structure containing a
callback function.
PB_HTYPE_POINTER 0x80 For required, optional and array non-scalar
fields, uses a pointer type (char* for strings,
pb_bytes_t for bytes, or a pointer for
submessages).
==================== ===== ================================================
pb_field_t
@@ -79,6 +83,16 @@ An byte array with a field for storing the length::
In an actual array, the length of *bytes* may be different.
pb_bytes_t
----------
A byte array with fields for storing the allocated and current lengths::
typedef struct {
size_t alloced;
size_t size;
uint8_t *bytes;
} pb_bytes_array_t;
pb_callback_t
-------------
Part of a message structure, for fields with type PB_HTYPE_CALLBACK::
@@ -368,6 +382,22 @@ In addition to EOF, the pb_decode implementation supports terminating a message
For optional fields, this function applies the default value and clears the corresponding bit in *has_fields* if the field is not present.
pb_clean
--------
Release and clear all the unused pointers of a structure. ::
bool pb_clean(const pb_message_t *msg, void *dest_struct);
:msg: A message descriptor. Usually autogenerated.
:dest_struct: Pointer to structure to be cleaned.
:returns: True on success, false if one of the unused message fields has a pointer type that this function cannot handle.
For each string or submessage with pointer type, this function calls *free()* on the pointer and sets the pointer to NULL.
For bytes fields with pointer type, this function calls *free()* on the allocated data, nulls that pointer and zeros the size fields in the generated pb_bytes_t structure.
For repeated fields, it applies the corresponding operation above to each unused element of the generated array.
.. sidebar:: Field decoders
The functions with names beginning with *pb_dec_* are called field decoders. Each PB_LTYPE has an own field decoder, which handles translating from Protocol Buffers data to C data.