Compare commits

...

40 Commits

Author SHA1 Message Date
Petteri Aimonen
ef422656a5 Fix oneof submessage initialization bug.
Update issue 149
Status: FixedInGit
2015-03-07 10:25:09 +02:00
Petteri Aimonen
5c16a116ec Better error messages for syntax errors in .options file 2015-02-26 18:16:25 +02:00
Petteri Aimonen
4a6580726e Fix generator error when long_names:false is combined with Oneofs.
Update issue 147
Status: FixedInGit
2015-02-26 17:33:36 +02:00
Petteri Aimonen
651e97456b Include libprotobuf in linux binary package.
Previously this got included by bbfreeze, but apparently no more.

Update issue 146
Status: FixedInGit
2015-02-22 15:28:26 +02:00
Petteri Aimonen
b836ac29dd Lower required CMake version in example 2015-02-13 18:57:46 +02:00
Petteri Aimonen
93364463ac Update cmake_simple example readme 2015-02-13 18:42:46 +02:00
Oliver Lee
7c00b90910 Add simple example built with CMake 2015-02-13 18:42:46 +02:00
Oliver Lee
02bd49bc93 Fix search for Python 2 with CMake
Do not assume that Python has already been found by CMake. Fix value of
CMake variable PYTHON_EXECUTABLE if Python 3 was found. Change minimum
supported Python version to 2.6.

This fixes a bug introduced by this commit:
d8d3b75e2e
2015-02-13 18:42:46 +02:00
Oliver Lee
d8d3b75e2e Updates for the CMake rule file.
1) Search explicitly for python2.7

In systems where python3 is default or in build cases where the user has
already searched for and found python3 in CMake, store the python3
executable and search for python2.7.

2) Generate nanopb core protobuf files with CMake

Generate python output files used in turn by the nanopb generator
script. This removes the requirement of manually calling 'make' in the
nanopb/generator/proto directory.

3) Use nanopb options file if it exists

Look for nanopb options file and use in protobuf source and header
generation if it exists. The options file must have the same name and
path as the proto file, excluding the extension.
2015-02-13 17:31:12 +02:00
Petteri Aimonen
25b92c5b4e Fix generator bug when oneof is first field in a message.
Added test case for the same.

Update issue 142
Status: FixedInGit
2015-01-27 17:47:25 +02:00
Petteri Aimonen
5aa8207ab1 Setting version to nanopb-0.3.3-dev 2015-01-24 17:40:42 +02:00
Petteri Aimonen
acd7291791 Publishing nanopb-0.3.2 2015-01-24 17:33:01 +02:00
Petteri Aimonen
d32d04ba10 Fix encoded_size #defines for oneof messages.
The sizes are represented as EncodedSize() instances, which cause
max() operation to sort them by address instead of value. This caused
pretty much random item to be selected for the maximum.

Update issue 141
Status: FixedInGit
2015-01-23 21:29:29 +02:00
Petteri Aimonen
0286a0746a Update changelog 2015-01-15 19:34:49 +02:00
Petteri Aimonen
24a45b0a9c Fix clang compiler warning in intsizes unit test. 2015-01-15 19:18:48 +02:00
Petteri Aimonen
e1c50496d9 Release memory when overwriting oneof fields.
Update issue 131
Status: FixedInGit
2015-01-15 18:58:08 +02:00
Petteri Aimonen
f4b3a1c202 Add oneofs to AllTypes test case 2015-01-11 19:47:27 +02:00
Petteri Aimonen
d2e023e3e5 Bugfixes for oneof support.
Fixes crashes / memory leaks when using pointer type fields.
Also fixes initialization of which_oneof fields.
2015-01-11 19:46:15 +02:00
Petteri Aimonen
8d12fecc7e New generator options for oneofs: allow skipping or generating as normal 'optional' fields.
The behaviour with no_unions:true is the same as of nanopb 0.3.1 and earlier.
2015-01-11 19:45:16 +02:00
Petteri Aimonen
8ef0392231 Fix generator error with OneOfs 2015-01-07 18:59:44 +02:00
Petteri Aimonen
fa444be424 Allow using 8/16/32/64 as values in int_size setting 2015-01-05 22:32:34 +02:00
Petteri Aimonen
cc3c8732fd Actually make the protoc version check work 2015-01-04 20:20:40 +02:00
Petteri Aimonen
7135e2797a Only run oneof test when protoc >= 2.6 is available 2015-01-04 20:00:37 +02:00
Petteri Aimonen
77a71ceb6d Fix build failure 2015-01-04 19:48:09 +02:00
Petteri Aimonen
7713d43bc3 Implement support for oneofs (C unions).
Basic test included, should probably add an oneof to the AllTypes test also.

Update issue 131
Status: Started
2015-01-04 19:39:37 +02:00
Petteri Aimonen
a0f0440394 Detect too large varint values when decoding.
Because Issue #139 now allows limiting integer fields, it is good
to check the values received from other protobuf libraries against
the lower limits.
2015-01-04 12:17:24 +02:00
Petteri Aimonen
50c67ecec4 Add int_size option for generator.
This allows overriding the integer field types to e.g. uint8_t for
saving RAM.

Update issue 139
Status: FixedInGit
2015-01-04 12:02:15 +02:00
Petteri Aimonen
b0d31468da Change PB_RETURN_ERROR() macro to avoid compiler warnings.
Update issue 140
Status: FixedInGit
2015-01-03 10:59:19 +02:00
Petteri Aimonen
7be7c7769f Fix build failure due to missing dependency in SConscript 2014-12-27 00:37:59 +02:00
Petteri Aimonen
88b2efe047 Fix memory leaks with PB_ENABLE_MALLOC and certain submessage type combinations.
There was a memory leak when:

1) A statically allocated submessage or
2) an extension field submessage

contained

A) a pointer-type field or
B) a submessage that further contained a pointer-type field.

This was because pb_release() didn't recurse into non-pointer fields.

Update issue 138
Status: FixedInGit
2014-12-26 23:27:35 +02:00
Petteri Aimonen
5008830488 Initialize also extension fields to defaults in pb_decode().
This makes the behaviour more consistent with non-extension fields,
and also makes sure that all 'found' fields of extensions are initially
false.
2014-12-26 23:27:35 +02:00
Petteri Aimonen
58643217b1 Fix bug in backwards_compatibility test case.
The memset() filled also the extensions field, which was just
waiting for a crash to happen.
2014-12-26 23:27:34 +02:00
Petteri Aimonen
1515cfb5c2 Add testcase for releasing memory in submessages/extensions 2014-12-26 23:27:29 +02:00
Petteri Aimonen
418f7d88b3 Add support for POINTER type in extensions 2014-12-26 18:23:36 +02:00
Petteri Aimonen
980f899dd5 Include the field type in a comment for extension fields 2014-12-26 17:43:38 +02:00
Petteri Aimonen
0f3c0f79bc Move malloc support to tests/common directory 2014-12-26 17:34:45 +02:00
Petteri Aimonen
8a28b70351 Move malloc_wrappers.c to tests/common 2014-12-26 17:08:17 +02:00
Petteri Aimonen
e5cbee84e1 Verify build with protobuf-3.0.0, fix problems.
Also updated descriptor.proto from protobuf-3.0.0.
2014-12-22 22:52:36 +02:00
Petteri Aimonen
cfc517f36b Add compilation option to disable struct packing.
Update issue 136
Status: FixedInGit
2014-12-22 20:52:40 +02:00
Petteri Aimonen
baf44b367f Set version to nanopb-0.3.2-dev 2014-09-16 20:41:45 +03:00
54 changed files with 1782 additions and 322 deletions

View File

@@ -1,3 +1,14 @@
nanopb-0.3.2 (2015-01-24)
Fix memory leaks with PB_ENABLE_MALLOC with some submessage hierarchies (issue 138)
Implement support for oneofs (C unions). (issues 131, 141)
Add int_size option for generator (issue 139)
Add compilation option to disable struct packing. (issue 136)
Change PB_RETURN_ERROR() macro to avoid compiler warnings (issue 140)
Fix build problems with protoc 3.0.0
Add support for POINTER type in extensions
Initialize also extension fields to defaults in pb_decode().
Detect too large varint values when decoding.
nanopb-0.3.1 (2014-09-11)
Fix security issue due to size_t overflows. (issue 132)
Fix memory leak with duplicated fields and PB_ENABLE_MALLOC

View File

@@ -11,6 +11,25 @@ are included, in order to make it easier to find this document.
.. contents ::
Nanopb-0.3.2 (2015-01-24)
=========================
Add support for OneOfs
----------------------
**Rationale:** Previously nanopb did not support the *oneof* construct in
*.proto* files. Those fields were generated as regular *optional* fields.
**Changes:** OneOfs are now generated as C unions. Callback fields are not
supported inside oneof and generator gives an error.
**Required actions:** The generator option *no_unions* can be used to restore old
behaviour and to allow callbacks to be used. To use unions, one change is
needed: use *which_xxxx* field to detect which field is present, instead
of *has_xxxx*. Compare the value against *MyStruct_myfield_tag*.
**Error indications:** Generator error: "Callback fields inside of oneof are
not supported". Compiler error: "Message" has no member named "has_xxxx".
Nanopb-0.3.0 (2014-08-26)
=========================

View File

@@ -24,6 +24,9 @@ __BIG_ENDIAN__ Set this if your platform stores integers and
floats in big-endian format. Mixed-endian
systems (different layout for ints and floats)
are currently not supported.
PB_NO_PACKED_STRUCTS Disable packed structs. Increases RAM usage but
is necessary on some platforms that do not
support unaligned memory access.
PB_ENABLE_MALLOC Set this to enable dynamic allocation support
in the decoder.
PB_MAX_REQUIRED_FIELDS Maximum number of required fields to check for
@@ -73,6 +76,8 @@ The generator behaviour can be adjusted using these options, defined in the
max_size Allocated size for *bytes* and *string* fields.
max_count Allocated number of entries in arrays
(*repeated* fields).
int_size Override the integer type of a field.
(To use e.g. uint8_t to save RAM.)
type Type of the generated field. Default value
is *FT_DEFAULT*, which selects automatically.
You can use *FT_CALLBACK*, *FT_POINTER*,
@@ -85,6 +90,7 @@ long_names Prefix the enum name to the enum value in
packed_struct Make the generated structures packed.
NOTE: This cannot be used on CPUs that break
on unaligned accesses to variables.
skip_message Skip the whole message from generation.
============================ ================================================
These options can be defined for the .proto files before they are converted

View File

@@ -0,0 +1,17 @@
cmake_minimum_required(VERSION 2.8)
project(NANOPB_CMAKE_SIMPLE C)
set(NANOPB_SRC_ROOT_FOLDER ${CMAKE_CURRENT_SOURCE_DIR}/../..)
set(CMAKE_MODULE_PATH ${NANOPB_SRC_ROOT_FOLDER}/extra)
find_package(Nanopb REQUIRED)
include_directories(${NANOPB_INCLUDE_DIRS})
nanopb_generate_cpp(PROTO_SRCS PROTO_HDRS simple.proto)
include_directories(${CMAKE_CURRENT_BINARY_DIR})
#add_custom_target(generate_proto_sources DEPENDS ${PROTO_SRCS} ${PROTO_HDRS})
set_source_files_properties(${PROTO_SRCS} ${PROTO_HDRS}
PROPERTIES GENERATED TRUE)
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wall -Werror -g -O0")
add_executable(simple simple.c ${PROTO_SRCS} ${PROTO_HDRS})

View File

@@ -0,0 +1,18 @@
Nanopb example "simple" using CMake
=======================
This example is the same as the simple nanopb example but built using CMake.
Example usage
-------------
On Linux, create a build directory and then call cmake:
nanopb/examples/cmake_simple$ mkdir build
nanopb/examples/cmake_simple$ cd build/
nanopb/examples/cmake_simple/build$ cmake ..
nanopb/examples/cmake_simple/build$ make
After that, you can run it with the command: ./simple
On other platforms supported by CMake, refer to CMake instructions.

View File

@@ -0,0 +1,68 @@
#include <stdio.h>
#include <pb_encode.h>
#include <pb_decode.h>
#include "simple.pb.h"
int main()
{
/* This is the buffer where we will store our message. */
uint8_t buffer[128];
size_t message_length;
bool status;
/* Encode our message */
{
/* Allocate space on the stack to store the message data.
*
* Nanopb generates simple struct definitions for all the messages.
* - check out the contents of simple.pb.h! */
SimpleMessage message;
/* Create a stream that will write to our buffer. */
pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
/* Fill in the lucky number */
message.lucky_number = 13;
/* Now we are ready to encode the message! */
status = pb_encode(&stream, SimpleMessage_fields, &message);
message_length = stream.bytes_written;
/* Then just check for any errors.. */
if (!status)
{
printf("Encoding failed: %s\n", PB_GET_ERROR(&stream));
return 1;
}
}
/* Now we could transmit the message over network, store it in a file or
* wrap it to a pigeon's leg.
*/
/* But because we are lazy, we will just decode it immediately. */
{
/* Allocate space for the decoded message. */
SimpleMessage message;
/* Create a stream that reads from the buffer. */
pb_istream_t stream = pb_istream_from_buffer(buffer, message_length);
/* Now we are ready to decode the message. */
status = pb_decode(&stream, SimpleMessage_fields, &message);
/* Check for errors... */
if (!status)
{
printf("Decoding failed: %s\n", PB_GET_ERROR(&stream));
return 1;
}
/* Print the data contained in the message. */
printf("Your lucky number was %d!\n", message.lucky_number);
}
return 0;
}

View File

@@ -0,0 +1,7 @@
// A very simple protocol definition, consisting of only
// one message.
message SimpleMessage {
required int32 lucky_number = 1;
}

View File

@@ -128,10 +128,36 @@ function(NANOPB_GENERATE_CPP SRCS HDRS)
set(${SRCS})
set(${HDRS})
get_filename_component(GENERATOR_PATH ${NANOPB_GENERATOR_EXECUTABLE} PATH)
set(GENERATOR_CORE_DIR ${GENERATOR_PATH}/proto)
set(GENERATOR_CORE_SRC
${GENERATOR_CORE_DIR}/nanopb.proto
${GENERATOR_CORE_DIR}/plugin.proto)
set(GENERATOR_CORE_PYTHON_SRC)
foreach(FIL ${GENERATOR_CORE_SRC})
get_filename_component(ABS_FIL ${FIL} ABSOLUTE)
get_filename_component(FIL_WE ${FIL} NAME_WE)
set(output "${GENERATOR_CORE_DIR}/${FIL_WE}_pb2.py")
set(GENERATOR_CORE_PYTHON_SRC ${GENERATOR_CORE_PYTHON_SRC} ${output})
add_custom_command(
OUTPUT ${output}
COMMAND ${PROTOBUF_PROTOC_EXECUTABLE}
ARGS -I${GENERATOR_PATH}/proto
--python_out=${GENERATOR_CORE_DIR} ${ABS_FIL}
DEPENDS ${ABS_FIL}
VERBATIM)
endforeach()
foreach(FIL ${ARGN})
get_filename_component(ABS_FIL ${FIL} ABSOLUTE)
get_filename_component(FIL_WE ${FIL} NAME_WE)
get_filename_component(FIL_DIR ${FIL} PATH)
set(NANOPB_OPTIONS_FILE ${FIL_DIR}/${FIL_WE}.options)
set(NANOPB_OPTIONS)
if(EXISTS ${NANOPB_OPTIONS_FILE})
set(NANOPB_OPTIONS -f ${NANOPB_OPTIONS_FILE})
endif()
list(APPEND ${SRCS} "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.c")
list(APPEND ${HDRS} "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.h")
@@ -139,16 +165,18 @@ function(NANOPB_GENERATE_CPP SRCS HDRS)
add_custom_command(
OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb"
COMMAND ${PROTOBUF_PROTOC_EXECUTABLE}
ARGS -I${GENERATOR_PATH} -I${CMAKE_CURRENT_BINARY_DIR} ${_nanobp_include_path} -o${FIL_WE}.pb ${ABS_FIL}
DEPENDS ${ABS_FIL}
ARGS -I${GENERATOR_PATH} -I${GENERATOR_CORE_DIR}
-I${CMAKE_CURRENT_BINARY_DIR} ${_nanobp_include_path}
-o${FIL_WE}.pb ${ABS_FIL}
DEPENDS ${ABS_FIL} ${GENERATOR_CORE_PYTHON_SRC}
COMMENT "Running C++ protocol buffer compiler on ${FIL}"
VERBATIM )
add_custom_command(
OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.c"
"${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.h"
COMMAND python
ARGS ${NANOPB_GENERATOR_EXECUTABLE} ${FIL_WE}.pb
COMMAND ${PYTHON2_EXECUTABLE}
ARGS ${NANOPB_GENERATOR_EXECUTABLE} ${FIL_WE}.pb ${NANOPB_OPTIONS}
DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb"
COMMENT "Running nanopb generator on ${FIL_WE}.pb"
VERBATIM )
@@ -216,6 +244,18 @@ find_file(NANOPB_GENERATOR_EXECUTABLE
)
mark_as_advanced(NANOPB_GENERATOR_EXECUTABLE)
# If python3 has already been found, save it and look for python2.6
if(${PYTHON_VERSION_MAJOR} AND ${PYTHON_VERSION_MAJOR} EQUAL 3)
set(PYTHON3_EXECUTABLE ${PYTHON_EXECUTABLE})
set(PYTHON_EXECUTABLE PYTHON_EXECUTABLE-NOTFOUND)
find_package(PythonInterp 2.6 REQUIRED)
set(PYTHON2_EXECUTABLE ${PYTHON_EXECUTABLE})
set(PYTHON_EXECUTABLE ${PYTHON3_EXECUTABLE})
else()
find_package(PythonInterp 2.6 REQUIRED)
set(PYTHON2_EXECUTABLE ${PYTHON_EXECUTABLE})
endif()
include(FindPackageHandleStandardArgs)
FIND_PACKAGE_HANDLE_STANDARD_ARGS(NANOPB DEFAULT_MSG
NANOPB_INCLUDE_DIRS

View File

@@ -1,7 +1,7 @@
#!/usr/bin/python
'''Generate header file for nanopb from a ProtoBuf FileDescriptorSet.'''
nanopb_version = "nanopb-0.3.1"
nanopb_version = "nanopb-0.3.3-dev"
import sys
@@ -44,22 +44,30 @@ except:
import time
import os.path
# Values are tuple (c type, pb type, encoded size)
# Values are tuple (c type, pb type, encoded size, int_size_allowed)
FieldD = descriptor.FieldDescriptorProto
datatypes = {
FieldD.TYPE_BOOL: ('bool', 'BOOL', 1),
FieldD.TYPE_DOUBLE: ('double', 'DOUBLE', 8),
FieldD.TYPE_FIXED32: ('uint32_t', 'FIXED32', 4),
FieldD.TYPE_FIXED64: ('uint64_t', 'FIXED64', 8),
FieldD.TYPE_FLOAT: ('float', 'FLOAT', 4),
FieldD.TYPE_INT32: ('int32_t', 'INT32', 10),
FieldD.TYPE_INT64: ('int64_t', 'INT64', 10),
FieldD.TYPE_SFIXED32: ('int32_t', 'SFIXED32', 4),
FieldD.TYPE_SFIXED64: ('int64_t', 'SFIXED64', 8),
FieldD.TYPE_SINT32: ('int32_t', 'SINT32', 5),
FieldD.TYPE_SINT64: ('int64_t', 'SINT64', 10),
FieldD.TYPE_UINT32: ('uint32_t', 'UINT32', 5),
FieldD.TYPE_UINT64: ('uint64_t', 'UINT64', 10)
FieldD.TYPE_BOOL: ('bool', 'BOOL', 1, False),
FieldD.TYPE_DOUBLE: ('double', 'DOUBLE', 8, False),
FieldD.TYPE_FIXED32: ('uint32_t', 'FIXED32', 4, False),
FieldD.TYPE_FIXED64: ('uint64_t', 'FIXED64', 8, False),
FieldD.TYPE_FLOAT: ('float', 'FLOAT', 4, False),
FieldD.TYPE_INT32: ('int32_t', 'INT32', 10, True),
FieldD.TYPE_INT64: ('int64_t', 'INT64', 10, True),
FieldD.TYPE_SFIXED32: ('int32_t', 'SFIXED32', 4, False),
FieldD.TYPE_SFIXED64: ('int64_t', 'SFIXED64', 8, False),
FieldD.TYPE_SINT32: ('int32_t', 'SINT32', 5, True),
FieldD.TYPE_SINT64: ('int64_t', 'SINT64', 10, True),
FieldD.TYPE_UINT32: ('uint32_t', 'UINT32', 5, True),
FieldD.TYPE_UINT64: ('uint64_t', 'UINT64', 10, True)
}
# Integer size overrides (from .proto settings)
intsizes = {
nanopb_pb2.IS_8: 'int8_t',
nanopb_pb2.IS_16: 'int16_t',
nanopb_pb2.IS_32: 'int32_t',
nanopb_pb2.IS_64: 'int64_t',
}
class Names:
@@ -163,6 +171,7 @@ class Field:
'''desc is FieldDescriptorProto'''
self.tag = desc.number
self.struct_name = struct_name
self.union_name = None
self.name = desc.name
self.default = None
self.max_size = None
@@ -226,7 +235,13 @@ class Field:
# Decide the C data type to use in the struct.
if datatypes.has_key(desc.type):
self.ctype, self.pbtype, self.enc_size = datatypes[desc.type]
self.ctype, self.pbtype, self.enc_size, isa = datatypes[desc.type]
# Override the field size if user wants to use smaller integers
if isa and field_options.int_size != nanopb_pb2.IS_DEFAULT:
self.ctype = intsizes[field_options.int_size]
if desc.type == FieldD.TYPE_UINT32 or desc.type == FieldD.TYPE_UINT64:
self.ctype = 'u' + self.ctype;
elif desc.type == FieldD.TYPE_ENUM:
self.pbtype = 'ENUM'
self.ctype = names_from_type_name(desc.type_name)
@@ -286,57 +301,91 @@ class Field:
if self.pbtype == 'BYTES' and self.allocation == 'STATIC':
result = 'typedef PB_BYTES_ARRAY_T(%d) %s;\n' % (self.max_size, self.ctype)
else:
result = None
result = ''
return result
def get_initializer(self, null_init):
'''Return literal expression for this field's default value.'''
def get_dependencies(self):
'''Get list of type names used by this field.'''
if self.allocation == 'STATIC':
return [str(self.ctype)]
else:
return []
def get_initializer(self, null_init, inner_init_only = False):
'''Return literal expression for this field's default value.
null_init: If True, initialize to a 0 value instead of default from .proto
inner_init_only: If True, exclude initialization for any count/has fields
'''
inner_init = None
if self.pbtype == 'MESSAGE':
if null_init:
return '%s_init_zero' % self.ctype
inner_init = '%s_init_zero' % self.ctype
else:
return '%s_init_default' % self.ctype
if self.default is None or null_init:
inner_init = '%s_init_default' % self.ctype
elif self.default is None or null_init:
if self.pbtype == 'STRING':
return '""'
inner_init = '""'
elif self.pbtype == 'BYTES':
return '{0, {0}}'
inner_init = '{0, {0}}'
elif self.pbtype == 'ENUM':
return '(%s)0' % self.ctype
inner_init = '(%s)0' % self.ctype
else:
return '0'
default = str(self.default)
if self.pbtype == 'STRING':
default = default.encode('utf-8').encode('string_escape')
default = default.replace('"', '\\"')
default = '"' + default + '"'
elif self.pbtype == 'BYTES':
data = default.decode('string_escape')
data = ['0x%02x' % ord(c) for c in data]
if len(data) == 0:
default = '{0, {0}}'
inner_init = '0'
else:
if self.pbtype == 'STRING':
inner_init = self.default.encode('utf-8').encode('string_escape')
inner_init = inner_init.replace('"', '\\"')
inner_init = '"' + inner_init + '"'
elif self.pbtype == 'BYTES':
data = str(self.default).decode('string_escape')
data = ['0x%02x' % ord(c) for c in data]
if len(data) == 0:
inner_init = '{0, {0}}'
else:
inner_init = '{%d, {%s}}' % (len(data), ','.join(data))
elif self.pbtype in ['FIXED32', 'UINT32']:
inner_init = str(self.default) + 'u'
elif self.pbtype in ['FIXED64', 'UINT64']:
inner_init = str(self.default) + 'ull'
elif self.pbtype in ['SFIXED64', 'INT64']:
inner_init = str(self.default) + 'll'
else:
default = '{%d, {%s}}' % (len(data), ','.join(data))
elif self.pbtype in ['FIXED32', 'UINT32']:
default += 'u'
elif self.pbtype in ['FIXED64', 'UINT64']:
default += 'ull'
elif self.pbtype in ['SFIXED64', 'INT64']:
default += 'll'
inner_init = str(self.default)
return default
if inner_init_only:
return inner_init
outer_init = None
if self.allocation == 'STATIC':
if self.rules == 'REPEATED':
outer_init = '0, {'
outer_init += ', '.join([inner_init] * self.max_count)
outer_init += '}'
elif self.rules == 'OPTIONAL':
outer_init = 'false, ' + inner_init
else:
outer_init = inner_init
elif self.allocation == 'POINTER':
if self.rules == 'REPEATED':
outer_init = '0, NULL'
else:
outer_init = 'NULL'
elif self.allocation == 'CALLBACK':
if self.pbtype == 'EXTENSION':
outer_init = 'NULL'
else:
outer_init = '{{NULL}, NULL}'
return outer_init
def default_decl(self, declaration_only = False):
'''Return definition for this field's default value.'''
if self.default is None:
return None
ctype = self.ctype
default = self.get_initializer(False)
default = self.get_initializer(False, True)
array_decl = ''
if self.pbtype == 'STRING':
@@ -361,7 +410,13 @@ class Field:
'''Return the pb_field_t initializer to use in the constant array.
prev_field_name is the name of the previous field or None.
'''
result = ' PB_FIELD(%3d, ' % self.tag
if self.rules == 'ONEOF':
result = ' PB_ONEOF_FIELD(%s, ' % self.union_name
else:
result = ' PB_FIELD('
result += '%3d, ' % self.tag
result += '%-8s, ' % self.pbtype
result += '%s, ' % self.rules
result += '%-8s, ' % self.allocation
@@ -389,6 +444,8 @@ class Field:
if self.pbtype == 'MESSAGE':
if self.rules == 'REPEATED' and self.allocation == 'STATIC':
return 'pb_membersize(%s, %s[0])' % (self.struct_name, self.name)
elif self.rules == 'ONEOF':
return 'pb_membersize(%s, %s.%s)' % (self.struct_name, self.union_name, self.name)
else:
return 'pb_membersize(%s, %s)' % (self.struct_name, self.name)
@@ -463,7 +520,7 @@ class ExtensionRange(Field):
return ' pb_extension_t *extensions;'
def types(self):
return None
return ''
def tags(self):
return ''
@@ -498,7 +555,8 @@ class ExtensionField(Field):
msg +=' type of extension fields is currently supported. */\n'
return msg
return 'extern const pb_extension_type_t %s;\n' % self.fullname
return ('extern const pb_extension_type_t %s; /* field type: %s */\n' %
(self.fullname, str(self).strip()))
def extension_def(self):
'''Definition of the extension type in the .pb.c file'''
@@ -519,6 +577,84 @@ class ExtensionField(Field):
return result
# ---------------------------------------------------------------------------
# Generation of oneofs (unions)
# ---------------------------------------------------------------------------
class OneOf(Field):
def __init__(self, struct_name, oneof_desc):
self.struct_name = struct_name
self.name = oneof_desc.name
self.ctype = 'union'
self.fields = []
self.allocation = 'ONEOF'
self.default = None
self.rules = 'ONEOF'
def add_field(self, field):
if field.allocation == 'CALLBACK':
raise Exception("Callback fields inside of oneof are not supported"
+ " (field %s)" % field.name)
field.union_name = self.name
field.rules = 'ONEOF'
self.fields.append(field)
self.fields.sort(key = lambda f: f.tag)
# Sort by the lowest tag number inside union
self.tag = min([f.tag for f in self.fields])
def __cmp__(self, other):
return cmp(self.tag, other.tag)
def __str__(self):
result = ''
if self.fields:
result += ' pb_size_t which_' + self.name + ";\n"
result += ' union {\n'
for f in self.fields:
result += ' ' + str(f).replace('\n', '\n ') + '\n'
result += ' } ' + self.name + ';'
return result
def types(self):
return ''.join([f.types() for f in self.fields])
def get_dependencies(self):
deps = []
for f in self.fields:
deps += f.get_dependencies()
return deps
def get_initializer(self, null_init):
return '0, {' + self.fields[0].get_initializer(null_init) + '}'
def default_decl(self, declaration_only = False):
return None
def tags(self):
return '\n'.join([f.tags() for f in self.fields])
def pb_field_t(self, prev_field_name):
result = ',\n'.join([f.pb_field_t(prev_field_name) for f in self.fields])
return result
def largest_field_value(self):
return max([f.largest_field_value() for f in self.fields])
def encoded_size(self, allmsgs):
largest = EncodedSize(0)
for f in self.fields:
size = f.encoded_size(allmsgs)
if size is None:
return None
elif size.symbols:
return None # Cannot resolve maximum of symbols
elif size.value > largest.value:
largest = size
return largest
# ---------------------------------------------------------------------------
# Generation of messages (structures)
# ---------------------------------------------------------------------------
@@ -528,11 +664,34 @@ class Message:
def __init__(self, names, desc, message_options):
self.name = names
self.fields = []
self.oneofs = {}
no_unions = []
if hasattr(desc, 'oneof_decl'):
for i, f in enumerate(desc.oneof_decl):
oneof_options = get_nanopb_suboptions(desc, message_options, self.name + f.name)
if oneof_options.no_unions:
no_unions.append(i) # No union, but add fields normally
elif oneof_options.type == nanopb_pb2.FT_IGNORE:
pass # No union and skip fields also
else:
oneof = OneOf(self.name, f)
self.oneofs[i] = oneof
self.fields.append(oneof)
for f in desc.field:
field_options = get_nanopb_suboptions(f, message_options, self.name + f.name)
if field_options.type != nanopb_pb2.FT_IGNORE:
self.fields.append(Field(self.name, f, field_options))
if field_options.type == nanopb_pb2.FT_IGNORE:
continue
field = Field(self.name, f, field_options)
if (hasattr(f, 'oneof_index') and
f.HasField('oneof_index') and
f.oneof_index not in no_unions):
if f.oneof_index in self.oneofs:
self.oneofs[f.oneof_index].add_field(field)
else:
self.fields.append(field)
if len(desc.extension_range) > 0:
field_options = get_nanopb_suboptions(desc, message_options, self.name + 'extensions')
@@ -546,7 +705,10 @@ class Message:
def get_dependencies(self):
'''Get list of type names that this structure refers to.'''
return [str(field.ctype) for field in self.fields if field.allocation == 'STATIC']
deps = []
for f in self.fields:
deps += f.get_dependencies()
return deps
def __str__(self):
result = 'typedef struct _%s {\n' % self.name
@@ -571,39 +733,15 @@ class Message:
return result
def types(self):
result = ""
for field in self.fields:
types = field.types()
if types is not None:
result += types + '\n'
return result
return ''.join([f.types() for f in self.fields])
def get_initializer(self, null_init):
if not self.ordered_fields:
return '{0}'
parts = []
for field in self.ordered_fields:
if field.allocation == 'STATIC':
if field.rules == 'REPEATED':
parts.append('0')
parts.append('{'
+ ', '.join([field.get_initializer(null_init)] * field.max_count)
+ '}')
elif field.rules == 'OPTIONAL':
parts.append('false')
parts.append(field.get_initializer(null_init))
else:
parts.append(field.get_initializer(null_init))
elif field.allocation == 'POINTER':
if field.rules == 'REPEATED':
parts.append('0')
parts.append('NULL')
elif field.allocation == 'CALLBACK':
if field.pbtype == 'EXTENSION':
parts.append('NULL')
else:
parts.append('{{NULL}, NULL}')
parts.append(field.get_initializer(null_init))
return '{' + ', '.join(parts) + '}'
def default_decl(self, declaration_only = False):
@@ -614,18 +752,39 @@ class Message:
result += default + '\n'
return result
def count_required_fields(self):
'''Returns number of required fields inside this message'''
count = 0
for f in self.fields:
if not isinstance(f, OneOf):
if f.rules == 'REQUIRED':
count += 1
return count
def count_all_fields(self):
count = 0
for f in self.fields:
if isinstance(f, OneOf):
count += len(f.fields)
else:
count += 1
return count
def fields_declaration(self):
result = 'extern const pb_field_t %s_fields[%d];' % (self.name, len(self.fields) + 1)
result = 'extern const pb_field_t %s_fields[%d];' % (self.name, self.count_all_fields() + 1)
return result
def fields_definition(self):
result = 'const pb_field_t %s_fields[%d] = {\n' % (self.name, len(self.fields) + 1)
result = 'const pb_field_t %s_fields[%d] = {\n' % (self.name, self.count_all_fields() + 1)
prev = None
for field in self.ordered_fields:
result += field.pb_field_t(prev)
result += ',\n'
prev = field.name
if isinstance(field, OneOf):
prev = field.name + '.' + field.fields[-1].name
else:
prev = field.name
result += ' PB_LAST_FIELD\n};'
return result
@@ -879,9 +1038,8 @@ def generate_source(headername, enums, messages, extensions, options):
# Add checks for numeric limits
if messages:
count_required_fields = lambda m: len([f for f in msg.fields if f.rules == 'REQUIRED'])
largest_msg = max(messages, key = count_required_fields)
largest_count = count_required_fields(largest_msg)
largest_msg = max(messages, key = lambda m: m.count_required_fields())
largest_count = largest_msg.count_required_fields()
if largest_count > 64:
yield '\n/* Check that missing required fields will be properly detected */\n'
yield '#if PB_MAX_REQUIRED_FIELDS < %d\n' % largest_count
@@ -968,14 +1126,28 @@ def read_options_file(infile):
[(namemask, options), ...]
'''
results = []
for line in infile:
for i, line in enumerate(infile):
line = line.strip()
if not line or line.startswith('//') or line.startswith('#'):
continue
parts = line.split(None, 1)
if len(parts) < 2:
sys.stderr.write("%s:%d: " % (infile.name, i + 1) +
"Option lines should have space between field name and options. " +
"Skipping line: '%s'\n" % line)
continue
opts = nanopb_pb2.NanoPBOptions()
text_format.Merge(parts[1], opts)
try:
text_format.Merge(parts[1], opts)
except Exception, e:
sys.stderr.write("%s:%d: " % (infile.name, i + 1) +
"Unparseable option line: '%s'. " % line +
"Error: %s\n" % str(e))
continue
results.append((parts[0], opts))
return results

View File

@@ -1,6 +1,6 @@
// Protocol Buffers - Google's data interchange format
// Copyright 2008 Google Inc. All rights reserved.
// http://code.google.com/p/protobuf/
// https://developers.google.com/protocol-buffers/
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
@@ -37,6 +37,7 @@
// without any other information (e.g. without reading its imports).
syntax = "proto2";
package google.protobuf;
option java_package = "com.google.protobuf";
@@ -74,10 +75,14 @@ message FileDescriptorProto {
optional FileOptions options = 8;
// This field contains optional information about the original source code.
// You may safely remove this entire field whithout harming runtime
// You may safely remove this entire field without harming runtime
// functionality of the descriptors -- the information is needed only by
// development tools.
optional SourceCodeInfo source_code_info = 9;
// The syntax of the proto file.
// The supported values are "proto2" and "proto3".
optional string syntax = 12;
}
// Describes a message type.
@@ -96,6 +101,8 @@ message DescriptorProto {
}
repeated ExtensionRange extension_range = 5;
repeated OneofDescriptorProto oneof_decl = 8;
optional MessageOptions options = 7;
}
@@ -143,7 +150,7 @@ message FieldDescriptorProto {
optional Label label = 4;
// If type_name is set, this need not be set. If both this and type_name
// are set, this must be either TYPE_ENUM or TYPE_MESSAGE.
// are set, this must be one of TYPE_ENUM, TYPE_MESSAGE or TYPE_GROUP.
optional Type type = 5;
// For message and enum types, this is the name of the type. If the name
@@ -164,9 +171,20 @@ message FieldDescriptorProto {
// TODO(kenton): Base-64 encode?
optional string default_value = 7;
// If set, gives the index of a oneof in the containing type's oneof_decl
// list. This field is a member of that oneof. Extensions of a oneof should
// not set this since the oneof to which they belong will be inferred based
// on the extension range containing the extension's field number.
optional int32 oneof_index = 9;
optional FieldOptions options = 8;
}
// Describes a oneof.
message OneofDescriptorProto {
optional string name = 1;
}
// Describes an enum type.
message EnumDescriptorProto {
optional string name = 1;
@@ -202,6 +220,11 @@ message MethodDescriptorProto {
optional string output_type = 3;
optional MethodOptions options = 4;
// Identifies if client streams multiple client messages
optional bool client_streaming = 5 [default=false];
// Identifies if server streams multiple server messages
optional bool server_streaming = 6 [default=false];
}
@@ -233,7 +256,7 @@ message MethodDescriptorProto {
// number. You can declare multiple options with only one extension number by
// putting them in a sub-message. See the Custom Options section of the docs
// for examples:
// http://code.google.com/apis/protocolbuffers/docs/proto.html#options
// https://developers.google.com/protocol-buffers/docs/proto#options
// If this turns out to be popular, a web service will be set up
// to automatically assign option numbers.
@@ -263,11 +286,26 @@ message FileOptions {
optional bool java_multiple_files = 10 [default=false];
// If set true, then the Java code generator will generate equals() and
// hashCode() methods for all messages defined in the .proto file. This is
// purely a speed optimization, as the AbstractMessage base class includes
// reflection-based implementations of these methods.
// hashCode() methods for all messages defined in the .proto file.
// - In the full runtime, this is purely a speed optimization, as the
// AbstractMessage base class includes reflection-based implementations of
// these methods.
//- In the lite runtime, setting this option changes the semantics of
// equals() and hashCode() to more closely match those of the full runtime;
// the generated methods compute their results based on field values rather
// than object identity. (Implementations should not assume that hashcodes
// will be consistent across runtimes or versions of the protocol compiler.)
optional bool java_generate_equals_and_hash = 20 [default=false];
// If set true, then the Java2 code generator will generate code that
// throws an exception whenever an attempt is made to assign a non-UTF-8
// byte sequence to a string field.
// Message reflection will do the same.
// However, an extension field still accepts non-UTF-8 byte sequences.
// This option has no effect on when used with the lite runtime.
optional bool java_string_check_utf8 = 27 [default=false];
// Generated classes can be optimized for speed or code size.
enum OptimizeMode {
SPEED = 1; // Generate complete code for parsing, serialization,
@@ -278,7 +316,10 @@ message FileOptions {
optional OptimizeMode optimize_for = 9 [default=SPEED];
// Sets the Go package where structs generated from this .proto will be
// placed. There is no default.
// placed. If omitted, the Go package will be derived from the following:
// - The basename of the package import path, if provided.
// - Otherwise, the package statement in the .proto file, if present.
// - Otherwise, the basename of the .proto file, without extension.
optional string go_package = 11;
@@ -287,7 +328,7 @@ message FileOptions {
// are not specific to any particular RPC system. They are generated by the
// main code generators in each language (without additional plugins).
// Generic services were the only kind of service generation supported by
// early versions of proto2.
// early versions of google.protobuf.
//
// Generic services are now considered deprecated in favor of using plugins
// that generate code specific to your particular RPC system. Therefore,
@@ -297,6 +338,18 @@ message FileOptions {
optional bool java_generic_services = 17 [default=false];
optional bool py_generic_services = 18 [default=false];
// Is this file deprecated?
// Depending on the target platform, this can emit Deprecated annotations
// for everything in the file, or it will be completely ignored; in the very
// least, this is a formalization for deprecating files.
optional bool deprecated = 23 [default=false];
// Enables the use of arenas for the proto messages in this file. This applies
// only to generated classes for C++.
optional bool cc_enable_arenas = 31 [default=false];
// The parser stores options it doesn't recognize here. See above.
repeated UninterpretedOption uninterpreted_option = 999;
@@ -330,6 +383,35 @@ message MessageOptions {
// from proto1 easier; new code should avoid fields named "descriptor".
optional bool no_standard_descriptor_accessor = 2 [default=false];
// Is this message deprecated?
// Depending on the target platform, this can emit Deprecated annotations
// for the message, or it will be completely ignored; in the very least,
// this is a formalization for deprecating messages.
optional bool deprecated = 3 [default=false];
// Whether the message is an automatically generated map entry type for the
// maps field.
//
// For maps fields:
// map<KeyType, ValueType> map_field = 1;
// The parsed descriptor looks like:
// message MapFieldEntry {
// option map_entry = true;
// optional KeyType key = 1;
// optional ValueType value = 2;
// }
// repeated MapFieldEntry map_field = 1;
//
// Implementations may choose not to generate the map_entry=true message, but
// use a native map in the target language to hold the keys and values.
// The reflection APIs in such implementions still need to work as
// if the field is a repeated message field.
//
// NOTE: Do not set the option in .proto files. Always use the maps syntax
// instead. The option should only be implicitly set by the proto compiler
// parser.
optional bool map_entry = 7;
// The parser stores options it doesn't recognize here. See above.
repeated UninterpretedOption uninterpreted_option = 999;
@@ -395,23 +477,11 @@ message FieldOptions {
// is a formalization for deprecating fields.
optional bool deprecated = 3 [default=false];
// EXPERIMENTAL. DO NOT USE.
// For "map" fields, the name of the field in the enclosed type that
// is the key for this map. For example, suppose we have:
// message Item {
// required string name = 1;
// required string value = 2;
// }
// message Config {
// repeated Item items = 1 [experimental_map_key="name"];
// }
// In this situation, the map key for Item will be set to "name".
// TODO: Fully-implement this, then remove the "experimental_" prefix.
optional string experimental_map_key = 9;
// For Google-internal migration only. Do not use.
optional bool weak = 10 [default=false];
// The parser stores options it doesn't recognize here. See above.
repeated UninterpretedOption uninterpreted_option = 999;
@@ -421,9 +491,15 @@ message FieldOptions {
message EnumOptions {
// Set this option to false to disallow mapping different tag names to a same
// Set this option to true to allow mapping different tag names to the same
// value.
optional bool allow_alias = 2 [default=true];
optional bool allow_alias = 2;
// Is this enum deprecated?
// Depending on the target platform, this can emit Deprecated annotations
// for the enum, or it will be completely ignored; in the very least, this
// is a formalization for deprecating enums.
optional bool deprecated = 3 [default=false];
// The parser stores options it doesn't recognize here. See above.
repeated UninterpretedOption uninterpreted_option = 999;
@@ -433,6 +509,12 @@ message EnumOptions {
}
message EnumValueOptions {
// Is this enum value deprecated?
// Depending on the target platform, this can emit Deprecated annotations
// for the enum value, or it will be completely ignored; in the very least,
// this is a formalization for deprecating enum values.
optional bool deprecated = 1 [default=false];
// The parser stores options it doesn't recognize here. See above.
repeated UninterpretedOption uninterpreted_option = 999;
@@ -447,6 +529,12 @@ message ServiceOptions {
// we were already using them long before we decided to release Protocol
// Buffers.
// Is this service deprecated?
// Depending on the target platform, this can emit Deprecated annotations
// for the service, or it will be completely ignored; in the very least,
// this is a formalization for deprecating services.
optional bool deprecated = 33 [default=false];
// The parser stores options it doesn't recognize here. See above.
repeated UninterpretedOption uninterpreted_option = 999;
@@ -461,6 +549,12 @@ message MethodOptions {
// we were already using them long before we decided to release Protocol
// Buffers.
// Is this method deprecated?
// Depending on the target platform, this can emit Deprecated annotations
// for the method, or it will be completely ignored; in the very least,
// this is a formalization for deprecating methods.
optional bool deprecated = 33 [default=false];
// The parser stores options it doesn't recognize here. See above.
repeated UninterpretedOption uninterpreted_option = 999;

View File

@@ -5,6 +5,7 @@
// These are used by nanopb to generate statically allocable structures
// for memory-limited environments.
syntax = "proto2";
import "google/protobuf/descriptor.proto";
option java_package = "fi.kapsi.koti.jpa.nanopb";
@@ -17,6 +18,14 @@ enum FieldType {
FT_IGNORE = 3; // Ignore the field completely.
}
enum IntSize {
IS_DEFAULT = 0; // Default, 32/64bit based on type in .proto
IS_8 = 8;
IS_16 = 16;
IS_32 = 32;
IS_64 = 64;
}
// This is the inner options message, which basically defines options for
// a field. When it is used in message or file scope, it applies to all
// fields.
@@ -27,6 +36,10 @@ message NanoPBOptions {
// Allocated number of entries in arrays ('repeated' fields)
optional int32 max_count = 2;
// Size of integer fields. Can save some memory if you don't need
// full 32 bits for the value.
optional IntSize int_size = 7 [default = IS_DEFAULT];
// Force type of field (callback or static allocation)
optional FieldType type = 3 [default = FT_DEFAULT];
@@ -40,6 +53,9 @@ message NanoPBOptions {
// Skip this message
optional bool skip_message = 6 [default = false];
// Generate oneof fields as normal optional fields instead of union.
optional bool no_unions = 8 [default = false];
}
// Extensions to protoc 'Descriptor' type in order to define options

View File

@@ -1,6 +1,6 @@
// Protocol Buffers - Google's data interchange format
// Copyright 2008 Google Inc. All rights reserved.
// http://code.google.com/p/protobuf/
// https://developers.google.com/protocol-buffers/
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
@@ -44,7 +44,10 @@
// plugin should be named "protoc-gen-$NAME", and will then be used when the
// flag "--${NAME}_out" is passed to protoc.
syntax = "proto2";
package google.protobuf.compiler;
option java_package = "com.google.protobuf.compiler";
option java_outer_classname = "PluginProtos";
import "google/protobuf/descriptor.proto";

58
pb.h
View File

@@ -17,6 +17,10 @@
* stores the most-significant byte first. */
/* #define __BIG_ENDIAN__ 1 */
/* Define this if your CPU / compiler combination does not support
* unaligned memory access to packed structures. */
/* #define PB_NO_PACKED_STRUCTS 1 */
/* Increase the number of required fields that are tracked.
* A compiler warning will tell if you need this. */
/* #define PB_MAX_REQUIRED_FIELDS 256 */
@@ -46,7 +50,7 @@
/* Version of the nanopb library. Just in case you want to check it in
* your own program. */
#define NANOPB_VERSION nanopb-0.3.1
#define NANOPB_VERSION nanopb-0.3.3-dev
/* Include all the system headers needed by nanopb. You will need the
* definitions of the following:
@@ -75,7 +79,12 @@
/* Macro for defining packed structures (compiler dependent).
* This just reduces memory requirements, but is not required.
*/
#if defined(__GNUC__) || defined(__clang__)
#if defined(PB_NO_PACKED_STRUCTS)
/* Disable struct packing */
# define PB_PACKED_STRUCT_START
# define PB_PACKED_STRUCT_END
# define pb_packed
#elif defined(__GNUC__) || defined(__clang__)
/* For GCC and clang */
# define PB_PACKED_STRUCT_START
# define PB_PACKED_STRUCT_END
@@ -174,6 +183,7 @@ typedef uint8_t pb_type_t;
#define PB_HTYPE_REQUIRED 0x00
#define PB_HTYPE_OPTIONAL 0x10
#define PB_HTYPE_REPEATED 0x20
#define PB_HTYPE_ONEOF 0x30
#define PB_HTYPE_MASK 0x30
/**** Field allocation types ****/
@@ -447,9 +457,11 @@ struct pb_extension_s {
0, \
pb_membersize(st, m), 0, ptr}
#define PB_OPTEXT_POINTER(tag, st, m, fd, ltype, ptr) \
PB_OPTIONAL_POINTER(tag, st, m, fd, ltype, ptr)
#define PB_OPTEXT_CALLBACK(tag, st, m, fd, ltype, ptr) \
{tag, PB_ATYPE_CALLBACK | PB_HTYPE_OPTIONAL | ltype, \
0, 0, pb_membersize(st, m), 0, ptr}
PB_OPTIONAL_CALLBACK(tag, st, m, fd, ltype, ptr)
/* The mapping from protobuf types to LTYPEs is done using these macros. */
#define PB_LTYPE_MAP_BOOL PB_LTYPE_VARINT
@@ -491,28 +503,44 @@ struct pb_extension_s {
PB_DATAOFFSET_ ## placement(message, field, prevfield), \
PB_LTYPE_MAP_ ## type, ptr)
/* Field description for oneof fields. This requires taking into account the
* union name also, that's why a separate set of macros is needed.
*/
#define PB_ONEOF_STATIC(u, tag, st, m, fd, ltype, ptr) \
{tag, PB_ATYPE_STATIC | PB_HTYPE_ONEOF | ltype, \
fd, pb_delta(st, which_ ## u, u.m), \
pb_membersize(st, u.m), 0, ptr}
#define PB_ONEOF_POINTER(u, tag, st, m, fd, ltype, ptr) \
{tag, PB_ATYPE_POINTER | PB_HTYPE_ONEOF | ltype, \
fd, pb_delta(st, which_ ## u, u.m), \
pb_membersize(st, u.m[0]), 0, ptr}
#define PB_ONEOF_FIELD(union_name, tag, type, rules, allocation, placement, message, field, prevfield, ptr) \
PB_ ## rules ## _ ## allocation(union_name, tag, message, field, \
PB_DATAOFFSET_ ## placement(message, union_name.field, prevfield), \
PB_LTYPE_MAP_ ## type, ptr)
/* These macros are used for giving out error messages.
* They are mostly a debugging aid; the main error information
* is the true/false return value from functions.
* Some code space can be saved by disabling the error
* messages if not used.
*
* PB_SET_ERROR() sets the error message if none has been set yet.
* msg must be a constant string literal.
* PB_GET_ERROR() always returns a pointer to a string.
* PB_RETURN_ERROR() sets the error and returns false from current
* function.
*/
#ifdef PB_NO_ERRMSG
#define PB_RETURN_ERROR(stream,msg) \
do {\
PB_UNUSED(stream); \
return false; \
} while(0)
#define PB_SET_ERROR(stream, msg) PB_UNUSED(stream)
#define PB_GET_ERROR(stream) "(errmsg disabled)"
#else
#define PB_RETURN_ERROR(stream,msg) \
do {\
if ((stream)->errmsg == NULL) \
(stream)->errmsg = (msg); \
return false; \
} while(0)
#define PB_SET_ERROR(stream, msg) (stream->errmsg = (stream)->errmsg ? (stream)->errmsg : (msg))
#define PB_GET_ERROR(stream) ((stream)->errmsg ? (stream)->errmsg : "(none)")
#endif
#define PB_RETURN_ERROR(stream, msg) return PB_SET_ERROR(stream, msg), false
#endif

View File

@@ -41,8 +41,15 @@ bool pb_field_iter_next(pb_field_iter_t *iter)
/* Increment the pointers based on previous field size */
size_t prev_size = prev_field->data_size;
if (PB_ATYPE(prev_field->type) == PB_ATYPE_STATIC &&
PB_HTYPE(prev_field->type) == PB_HTYPE_REPEATED)
if (PB_HTYPE(prev_field->type) == PB_HTYPE_ONEOF &&
PB_HTYPE(iter->pos->type) == PB_HTYPE_ONEOF)
{
/* Don't advance pointers inside unions */
prev_size = 0;
iter->pData = (char*)iter->pData - prev_field->data_offset;
}
else if (PB_ATYPE(prev_field->type) == PB_ATYPE_STATIC &&
PB_HTYPE(prev_field->type) == PB_HTYPE_REPEATED)
{
/* In static arrays, the data_size tells the size of a single entry and
* array_size is the number of entries */
@@ -54,7 +61,7 @@ bool pb_field_iter_next(pb_field_iter_t *iter)
* The data_size only applies to the dynamically allocated area. */
prev_size = sizeof(void*);
}
if (PB_HTYPE(prev_field->type) == PB_HTYPE_REQUIRED)
{
/* Count the required fields, in order to check their presence in the

View File

@@ -29,9 +29,11 @@ static bool checkreturn read_raw_value(pb_istream_t *stream, pb_wire_type_t wire
static bool checkreturn decode_static_field(pb_istream_t *stream, pb_wire_type_t wire_type, pb_field_iter_t *iter);
static bool checkreturn decode_callback_field(pb_istream_t *stream, pb_wire_type_t wire_type, pb_field_iter_t *iter);
static bool checkreturn decode_field(pb_istream_t *stream, pb_wire_type_t wire_type, pb_field_iter_t *iter);
static void iter_from_extension(pb_field_iter_t *iter, pb_extension_t *extension);
static bool checkreturn default_extension_decoder(pb_istream_t *stream, pb_extension_t *extension, uint32_t tag, pb_wire_type_t wire_type);
static bool checkreturn decode_extension(pb_istream_t *stream, uint32_t tag, pb_wire_type_t wire_type, pb_field_iter_t *iter);
static bool checkreturn find_extension_field(pb_field_iter_t *iter);
static void pb_field_set_to_default(pb_field_iter_t *iter);
static void pb_message_set_to_defaults(const pb_field_t fields[], void *dest_struct);
static bool checkreturn pb_dec_varint(pb_istream_t *stream, const pb_field_t *field, void *dest);
static bool checkreturn pb_dec_uvarint(pb_istream_t *stream, const pb_field_t *field, void *dest);
@@ -46,6 +48,7 @@ static bool checkreturn pb_skip_string(pb_istream_t *stream);
#ifdef PB_ENABLE_MALLOC
static bool checkreturn allocate_field(pb_istream_t *stream, void *pData, size_t data_size, size_t array_size);
static bool checkreturn pb_release_union_field(pb_istream_t *stream, pb_field_iter_t *iter);
static void pb_release_single_field(const pb_field_iter_t *iter);
#endif
@@ -391,6 +394,14 @@ static bool checkreturn decode_static_field(pb_istream_t *stream, pb_wire_type_t
return func(stream, iter->pos, pItem);
}
case PB_HTYPE_ONEOF:
*(pb_size_t*)iter->pSize = iter->pos->tag;
if (PB_LTYPE(type) == PB_LTYPE_SUBMESSAGE)
{
pb_message_set_to_defaults((const pb_field_t*)iter->pos->ptr, iter->pData);
}
return func(stream, iter->pos, iter->pData);
default:
PB_RETURN_ERROR(stream, "invalid field type");
}
@@ -468,6 +479,7 @@ static bool checkreturn decode_pointer_field(pb_istream_t *stream, pb_wire_type_
{
case PB_HTYPE_REQUIRED:
case PB_HTYPE_OPTIONAL:
case PB_HTYPE_ONEOF:
if (PB_LTYPE(type) == PB_LTYPE_SUBMESSAGE &&
*(void**)iter->pData != NULL)
{
@@ -475,6 +487,11 @@ static bool checkreturn decode_pointer_field(pb_istream_t *stream, pb_wire_type_
pb_release_single_field(iter);
}
if (PB_HTYPE(type) == PB_HTYPE_ONEOF)
{
*(pb_size_t*)iter->pSize = iter->pos->tag;
}
if (PB_LTYPE(type) == PB_LTYPE_STRING ||
PB_LTYPE(type) == PB_LTYPE_BYTES)
{
@@ -560,7 +577,7 @@ static bool checkreturn decode_pointer_field(pb_istream_t *stream, pb_wire_type_
initialize_pointer_field(pItem, iter);
return func(stream, iter->pos, pItem);
}
default:
PB_RETURN_ERROR(stream, "invalid field type");
}
@@ -616,6 +633,16 @@ static bool checkreturn decode_callback_field(pb_istream_t *stream, pb_wire_type
static bool checkreturn decode_field(pb_istream_t *stream, pb_wire_type_t wire_type, pb_field_iter_t *iter)
{
#ifdef PB_ENABLE_MALLOC
/* When decoding an oneof field, check if there is old data that must be
* released first. */
if (PB_HTYPE(iter->pos->type) == PB_HTYPE_ONEOF)
{
if (!pb_release_union_field(stream, iter))
return false;
}
#endif
switch (PB_ATYPE(iter->pos->type))
{
case PB_ATYPE_STATIC:
@@ -632,6 +659,25 @@ static bool checkreturn decode_field(pb_istream_t *stream, pb_wire_type_t wire_t
}
}
static void iter_from_extension(pb_field_iter_t *iter, pb_extension_t *extension)
{
/* Fake a field iterator for the extension field.
* It is not actually safe to advance this iterator, but decode_field
* will not even try to. */
const pb_field_t *field = (const pb_field_t*)extension->type->arg;
(void)pb_field_iter_begin(iter, field, extension->dest);
iter->pData = extension->dest;
iter->pSize = &extension->found;
if (PB_ATYPE(field->type) == PB_ATYPE_POINTER)
{
/* For pointer extensions, the pointer is stored directly
* in the extension structure. This avoids having an extra
* indirection. */
iter->pData = &extension->dest;
}
}
/* Default handler for extension fields. Expects a pb_field_t structure
* in extension->type->arg. */
static bool checkreturn default_extension_decoder(pb_istream_t *stream,
@@ -643,13 +689,8 @@ static bool checkreturn default_extension_decoder(pb_istream_t *stream,
if (field->tag != tag)
return true;
/* Fake a field iterator for the extension field.
* It is not actually safe to advance this iterator, but decode_field
* will not even try to. */
(void)pb_field_iter_begin(&iter, field, extension->dest);
iter.pData = extension->dest;
iter.pSize = &extension->found;
iter_from_extension(&iter, extension);
extension->found = true;
return decode_field(stream, wire_type, &iter);
}
@@ -695,6 +736,78 @@ static bool checkreturn find_extension_field(pb_field_iter_t *iter)
}
/* Initialize message fields to default values, recursively */
static void pb_field_set_to_default(pb_field_iter_t *iter)
{
pb_type_t type;
type = iter->pos->type;
if (PB_LTYPE(type) == PB_LTYPE_EXTENSION)
{
pb_extension_t *ext = *(pb_extension_t* const *)iter->pData;
while (ext != NULL)
{
pb_field_iter_t ext_iter;
ext->found = false;
iter_from_extension(&ext_iter, ext);
pb_field_set_to_default(&ext_iter);
ext = ext->next;
}
}
else if (PB_ATYPE(type) == PB_ATYPE_STATIC)
{
bool init_data = true;
if (PB_HTYPE(type) == PB_HTYPE_OPTIONAL)
{
/* Set has_field to false. Still initialize the optional field
* itself also. */
*(bool*)iter->pSize = false;
}
else if (PB_HTYPE(type) == PB_HTYPE_REPEATED ||
PB_HTYPE(type) == PB_HTYPE_ONEOF)
{
/* REPEATED: Set array count to 0, no need to initialize contents.
ONEOF: Set which_field to 0. */
*(pb_size_t*)iter->pSize = 0;
init_data = false;
}
if (init_data)
{
if (PB_LTYPE(iter->pos->type) == PB_LTYPE_SUBMESSAGE)
{
/* Initialize submessage to defaults */
pb_message_set_to_defaults((const pb_field_t *) iter->pos->ptr, iter->pData);
}
else if (iter->pos->ptr != NULL)
{
/* Initialize to default value */
memcpy(iter->pData, iter->pos->ptr, iter->pos->data_size);
}
else
{
/* Initialize to zeros */
memset(iter->pData, 0, iter->pos->data_size);
}
}
}
else if (PB_ATYPE(type) == PB_ATYPE_POINTER)
{
/* Initialize the pointer to NULL. */
*(void**)iter->pData = NULL;
/* Initialize array count to 0. */
if (PB_HTYPE(type) == PB_HTYPE_REPEATED ||
PB_HTYPE(type) == PB_HTYPE_ONEOF)
{
*(pb_size_t*)iter->pSize = 0;
}
}
else if (PB_ATYPE(type) == PB_ATYPE_CALLBACK)
{
/* Don't overwrite callback */
}
}
static void pb_message_set_to_defaults(const pb_field_t fields[], void *dest_struct)
{
pb_field_iter_t iter;
@@ -704,55 +817,7 @@ static void pb_message_set_to_defaults(const pb_field_t fields[], void *dest_str
do
{
pb_type_t type;
type = iter.pos->type;
if (PB_ATYPE(type) == PB_ATYPE_STATIC)
{
if (PB_HTYPE(type) == PB_HTYPE_OPTIONAL)
{
/* Set has_field to false. Still initialize the optional field
* itself also. */
*(bool*)iter.pSize = false;
}
else if (PB_HTYPE(type) == PB_HTYPE_REPEATED)
{
/* Set array count to 0, no need to initialize contents. */
*(pb_size_t*)iter.pSize = 0;
continue;
}
if (PB_LTYPE(iter.pos->type) == PB_LTYPE_SUBMESSAGE)
{
/* Initialize submessage to defaults */
pb_message_set_to_defaults((const pb_field_t *) iter.pos->ptr, iter.pData);
}
else if (iter.pos->ptr != NULL)
{
/* Initialize to default value */
memcpy(iter.pData, iter.pos->ptr, iter.pos->data_size);
}
else
{
/* Initialize to zeros */
memset(iter.pData, 0, iter.pos->data_size);
}
}
else if (PB_ATYPE(type) == PB_ATYPE_POINTER)
{
/* Initialize the pointer to NULL. */
*(void**)iter.pData = NULL;
/* Initialize array count to 0. */
if (PB_HTYPE(type) == PB_HTYPE_REPEATED)
{
*(pb_size_t*)iter.pSize = 0;
}
}
else if (PB_ATYPE(type) == PB_ATYPE_CALLBACK)
{
/* Don't overwrite callback */
}
pb_field_set_to_default(&iter);
} while (pb_field_iter_next(&iter));
}
@@ -886,11 +951,86 @@ bool pb_decode_delimited(pb_istream_t *stream, const pb_field_t fields[], void *
}
#ifdef PB_ENABLE_MALLOC
/* Given an oneof field, if there has already been a field inside this oneof,
* release it before overwriting with a different one. */
static bool pb_release_union_field(pb_istream_t *stream, pb_field_iter_t *iter)
{
pb_size_t old_tag = *(pb_size_t*)iter->pSize; /* Previous which_ value */
pb_size_t new_tag = iter->pos->tag; /* New which_ value */
if (old_tag == 0)
return true; /* Ok, no old data in union */
if (old_tag == new_tag)
return true; /* Ok, old data is of same type => merge */
/* Release old data. The find can fail if the message struct contains
* invalid data. */
if (!pb_field_iter_find(iter, old_tag))
PB_RETURN_ERROR(stream, "invalid union tag");
pb_release_single_field(iter);
/* Restore iterator to where it should be.
* This shouldn't fail unless the pb_field_t structure is corrupted. */
if (!pb_field_iter_find(iter, new_tag))
PB_RETURN_ERROR(stream, "iterator error");
return true;
}
static void pb_release_single_field(const pb_field_iter_t *iter)
{
pb_type_t type;
type = iter->pos->type;
if (PB_HTYPE(type) == PB_HTYPE_ONEOF)
{
if (*(pb_size_t*)iter->pSize != iter->pos->tag)
return; /* This is not the current field in the union */
}
/* Release anything contained inside an extension or submsg.
* This has to be done even if the submsg itself is statically
* allocated. */
if (PB_LTYPE(type) == PB_LTYPE_EXTENSION)
{
/* Release fields from all extensions in the linked list */
pb_extension_t *ext = *(pb_extension_t**)iter->pData;
while (ext != NULL)
{
pb_field_iter_t ext_iter;
iter_from_extension(&ext_iter, ext);
pb_release_single_field(&ext_iter);
ext = ext->next;
}
}
else if (PB_LTYPE(type) == PB_LTYPE_SUBMESSAGE)
{
/* Release fields in submessage or submsg array */
void *pItem = iter->pData;
pb_size_t count = 1;
if (PB_ATYPE(type) == PB_ATYPE_POINTER)
{
pItem = *(void**)iter->pData;
}
if (PB_HTYPE(type) == PB_HTYPE_REPEATED)
{
count = *(pb_size_t*)iter->pSize;
}
if (pItem)
{
while (count--)
{
pb_release((const pb_field_t*)iter->pos->ptr, pItem);
pItem = (uint8_t*)pItem + iter->pos->data_size;
}
}
}
if (PB_ATYPE(type) == PB_ATYPE_POINTER)
{
if (PB_HTYPE(type) == PB_HTYPE_REPEATED &&
@@ -905,28 +1045,12 @@ static void pb_release_single_field(const pb_field_iter_t *iter)
pb_free(*pItem);
*pItem++ = NULL;
}
*(pb_size_t*)iter->pSize = 0;
}
else if (PB_LTYPE(type) == PB_LTYPE_SUBMESSAGE)
if (PB_HTYPE(type) == PB_HTYPE_REPEATED)
{
/* Release fields in submessages */
void *pItem = *(void**)iter->pData;
if (pItem)
{
pb_size_t count = 1;
if (PB_HTYPE(type) == PB_HTYPE_REPEATED)
{
count = *(pb_size_t*)iter->pSize;
*(pb_size_t*)iter->pSize = 0;
}
while (count--)
{
pb_release((const pb_field_t*)iter->pos->ptr, pItem);
pItem = (uint8_t*)pItem + iter->pos->data_size;
}
}
/* We are going to release the array, so set the size to 0 */
*(pb_size_t*)iter->pSize = 0;
}
/* Release main item */
@@ -1010,49 +1134,75 @@ bool pb_decode_fixed64(pb_istream_t *stream, void *dest)
static bool checkreturn pb_dec_varint(pb_istream_t *stream, const pb_field_t *field, void *dest)
{
uint64_t value;
int64_t svalue;
int64_t clamped;
if (!pb_decode_varint(stream, &value))
return false;
/* See issue 97: Google's C++ protobuf allows negative varint values to
* be cast as int32_t, instead of the int64_t that should be used when
* encoding. Previous nanopb versions had a bug in encoding. In order to
* not break decoding of such messages, we cast <=32 bit fields to
* int32_t first to get the sign correct.
*/
if (field->data_size == 8)
svalue = (int64_t)value;
else
svalue = (int32_t)value;
switch (field->data_size)
{
case 1: *(int8_t*)dest = (int8_t)value; break;
case 2: *(int16_t*)dest = (int16_t)value; break;
case 4: *(int32_t*)dest = (int32_t)value; break;
case 8: *(int64_t*)dest = (int64_t)value; break;
case 1: clamped = *(int8_t*)dest = (int8_t)svalue; break;
case 2: clamped = *(int16_t*)dest = (int16_t)svalue; break;
case 4: clamped = *(int32_t*)dest = (int32_t)svalue; break;
case 8: clamped = *(int64_t*)dest = svalue; break;
default: PB_RETURN_ERROR(stream, "invalid data_size");
}
if (clamped != svalue)
PB_RETURN_ERROR(stream, "integer too large");
return true;
}
static bool checkreturn pb_dec_uvarint(pb_istream_t *stream, const pb_field_t *field, void *dest)
{
uint64_t value;
uint64_t value, clamped;
if (!pb_decode_varint(stream, &value))
return false;
switch (field->data_size)
{
case 4: *(uint32_t*)dest = (uint32_t)value; break;
case 8: *(uint64_t*)dest = value; break;
case 1: clamped = *(uint8_t*)dest = (uint8_t)value; break;
case 2: clamped = *(uint16_t*)dest = (uint16_t)value; break;
case 4: clamped = *(uint32_t*)dest = (uint32_t)value; break;
case 8: clamped = *(uint64_t*)dest = value; break;
default: PB_RETURN_ERROR(stream, "invalid data_size");
}
if (clamped != value)
PB_RETURN_ERROR(stream, "integer too large");
return true;
}
static bool checkreturn pb_dec_svarint(pb_istream_t *stream, const pb_field_t *field, void *dest)
{
int64_t value;
int64_t value, clamped;
if (!pb_decode_svarint(stream, &value))
return false;
switch (field->data_size)
{
case 4: *(int32_t*)dest = (int32_t)value; break;
case 8: *(int64_t*)dest = value; break;
case 1: clamped = *(int8_t*)dest = (int8_t)value; break;
case 2: clamped = *(int16_t*)dest = (int16_t)value; break;
case 4: clamped = *(int32_t*)dest = (int32_t)value; break;
case 8: clamped = *(int64_t*)dest = value; break;
default: PB_RETURN_ERROR(stream, "invalid data_size");
}
if (clamped != value)
PB_RETURN_ERROR(stream, "integer too large");
return true;
}

View File

@@ -250,6 +250,17 @@ static bool checkreturn encode_basic_field(pb_ostream_t *stream,
return false;
break;
case PB_HTYPE_ONEOF:
if (*(const pb_size_t*)pSize == field->tag)
{
if (!pb_encode_tag_for_field(stream, field))
return false;
if (!func(stream, field, pData))
return false;
}
break;
default:
PB_RETURN_ERROR(stream, "invalid field type");
}
@@ -302,7 +313,18 @@ static bool checkreturn default_extension_encoder(pb_ostream_t *stream,
const pb_extension_t *extension)
{
const pb_field_t *field = (const pb_field_t*)extension->type->arg;
return encode_field(stream, field, extension->dest);
if (PB_ATYPE(field->type) == PB_ATYPE_POINTER)
{
/* For pointer extensions, the pointer is stored directly
* in the extension structure. This avoids having an extra
* indirection. */
return encode_field(stream, field, &extension->dest);
}
else
{
return encode_field(stream, field, extension->dest);
}
}
/* Walk through all the registered extensions and give them a chance
@@ -556,7 +578,7 @@ static bool checkreturn pb_enc_varint(pb_ostream_t *stream, const pb_field_t *fi
int64_t value = 0;
/* Cases 1 and 2 are for compilers that have smaller types for bool
* or enums. */
* or enums, and for int_size option. */
switch (field->data_size)
{
case 1: value = *(const int8_t*)src; break;
@@ -575,6 +597,8 @@ static bool checkreturn pb_enc_uvarint(pb_ostream_t *stream, const pb_field_t *f
switch (field->data_size)
{
case 1: value = *(const uint8_t*)src; break;
case 2: value = *(const uint16_t*)src; break;
case 4: value = *(const uint32_t*)src; break;
case 8: value = *(const uint64_t*)src; break;
default: PB_RETURN_ERROR(stream, "invalid data_size");
@@ -589,6 +613,8 @@ static bool checkreturn pb_enc_svarint(pb_ostream_t *stream, const pb_field_t *f
switch (field->data_size)
{
case 1: value = *(const int8_t*)src; break;
case 2: value = *(const int16_t*)src; break;
case 4: value = *(const int32_t*)src; break;
case 8: value = *(const int64_t*)src; break;
default: PB_RETURN_ERROR(stream, "invalid data_size");

View File

@@ -72,6 +72,11 @@ if not env.GetOption('clean'):
else:
conf.env.Append(PROTOCPATH = '/usr/include')
# Check protoc version
status, output = conf.TryAction('$PROTOC --version > $TARGET')
if status:
conf.env['PROTOC_VERSION'] = output
# Check if libmudflap is available (only with GCC)
if 'gcc' in env['CC']:
if conf.CheckLib('mudflap'):
@@ -121,9 +126,6 @@ elif 'cl' in env['CC']:
# More strict checks on the nanopb core
env.Append(CORECFLAGS = '/W4')
# PB_RETURN_ERROR triggers C4127 because of while(0)
env.Append(CFLAGS = '/wd4127')
elif 'tcc' in env['CC']:
# Tiny C Compiler
env.Append(CFLAGS = '-Wall -Werror -g')
@@ -136,12 +138,18 @@ elif 'g++' in env['CXX'] or 'gcc' in env['CXX']:
env.Append(CXXFLAGS = '-g -Wall -Werror -Wextra -Wno-missing-field-initializers')
elif 'cl' in env['CXX']:
env.Append(CXXFLAGS = '/Zi /W2 /WX')
# Now include the SConscript files from all subdirectories
import os.path
env['VARIANT_DIR'] = 'build'
env['BUILD'] = '#' + env['VARIANT_DIR']
env['COMMON'] = '#' + env['VARIANT_DIR'] + '/common'
# Include common/SConscript first to make sure its exports are available
# to other SConscripts.
SConscript("common/SConscript", exports = 'env', variant_dir = env['VARIANT_DIR'] + '/common')
for subdir in Glob('*/SConscript') + Glob('regression/*/SConscript'):
if str(subdir).startswith("common"): continue
SConscript(subdir, exports = 'env', variant_dir = env['VARIANT_DIR'] + '/' + os.path.dirname(str(subdir)))

View File

@@ -101,6 +101,12 @@ message AllTypes {
optional MyEnum opt_enum = 57 [default = Second];
optional EmptyMessage opt_emptymsg = 58;
oneof oneof
{
SubMessage oneof_msg1 = 59;
EmptyMessage oneof_msg2 = 60;
}
// Check that extreme integer values are handled correctly
required Limits req_limits = 98;

View File

@@ -125,6 +125,8 @@ bool check_alltypes(pb_istream_t *stream, int mode)
TEST(alltypes.has_opt_enum == false);
TEST(alltypes.opt_enum == MyEnum_Second);
TEST(alltypes.has_opt_emptymsg == false);
TEST(alltypes.which_oneof == 0);
}
else
{
@@ -170,6 +172,10 @@ bool check_alltypes(pb_istream_t *stream, int mode)
TEST(alltypes.has_opt_enum == true);
TEST(alltypes.opt_enum == MyEnum_Truth);
TEST(alltypes.has_opt_emptymsg == true);
TEST(alltypes.which_oneof == AllTypes_oneof_msg1_tag);
TEST(strcmp(alltypes.oneof.oneof_msg1.substuff1, "4059") == 0);
TEST(alltypes.oneof.oneof_msg1.substuff2 == 4059);
}
TEST(alltypes.req_limits.int32_min == INT32_MIN);

View File

@@ -121,6 +121,10 @@ int main(int argc, char **argv)
alltypes.has_opt_enum = true;
alltypes.opt_enum = MyEnum_Truth;
alltypes.has_opt_emptymsg = true;
alltypes.which_oneof = AllTypes_oneof_msg1_tag;
strcpy(alltypes.oneof.oneof_msg1.substuff1, "4059");
alltypes.oneof.oneof_msg1.substuff2 = 4059;
}
alltypes.end = 1099;

View File

@@ -1,3 +1,4 @@
# Generate all fields as callbacks.
AllTypes.* type:FT_CALLBACK
SubMessage.substuff1 max_size:16
AllTypes.oneof no_unions:true

View File

@@ -214,6 +214,8 @@ bool check_alltypes(pb_istream_t *stream, int mode)
int64_t opt_sfixed64 = 3052;
double opt_double = 3053.0f;
SubMessage opt_submsg = {"3056", 3056};
SubMessage oneof_msg1 = {"4059", 4059};
/* Bind callbacks for required fields */
AllTypes alltypes;
@@ -392,6 +394,9 @@ bool check_alltypes(pb_istream_t *stream, int mode)
alltypes.opt_enum.arg = (void*)MyEnum_Truth;
alltypes.opt_emptymsg.funcs.decode = &read_emptymsg;
alltypes.oneof_msg1.funcs.decode = &read_submsg;
alltypes.oneof_msg1.arg = &oneof_msg1;
}
return pb_decode(stream, AllTypes_fields, &alltypes);

View File

@@ -202,6 +202,8 @@ int main(int argc, char **argv)
double opt_double = 3053.0f;
SubMessage opt_submsg = {"3056", 3056};
SubMessage oneof_msg1 = {"4059", 4059};
/* Bind callbacks for required fields */
AllTypes alltypes = {{{0}}};
@@ -372,6 +374,9 @@ int main(int argc, char **argv)
alltypes.opt_enum.arg = (void*)MyEnum_Truth;
alltypes.opt_emptymsg.funcs.encode = &write_emptymsg;
alltypes.oneof_msg1.funcs.encode = &write_submsg;
alltypes.oneof_msg1.arg = &oneof_msg1;
}
alltypes.end.funcs.encode = &write_varint;

View File

@@ -1,31 +1,22 @@
# Encode the AllTypes message using pointers for all fields, and verify the
# output against the normal AllTypes test case.
Import("env")
# We need our own pb_decode.o for the malloc support
env = env.Clone()
env.Append(CPPDEFINES = {'PB_ENABLE_MALLOC': 1});
# Disable libmudflap, because it will confuse valgrind
# and other memory leak detection tools.
if '-fmudflap' in env["CCFLAGS"]:
env["CCFLAGS"].remove("-fmudflap")
env["LINKFLAGS"].remove("-fmudflap")
env["LIBS"].remove("mudflap")
strict = env.Clone()
strict.Append(CFLAGS = strict['CORECFLAGS'])
strict.Object("pb_decode_with_malloc.o", "$NANOPB/pb_decode.c")
strict.Object("pb_encode_with_malloc.o", "$NANOPB/pb_encode.c")
strict.Object("pb_common_with_malloc.o", "$NANOPB/pb_common.c")
Import("env", "malloc_env")
c = Copy("$TARGET", "$SOURCE")
env.Command("alltypes.proto", "#alltypes/alltypes.proto", c)
env.NanopbProto(["alltypes", "alltypes.options"])
enc = env.Program(["encode_alltypes_pointer.c", "alltypes.pb.c", "pb_encode_with_malloc.o", "pb_common_with_malloc.o"])
dec = env.Program(["decode_alltypes_pointer.c", "alltypes.pb.c", "pb_decode_with_malloc.o", "pb_common_with_malloc.o"])
enc = malloc_env.Program(["encode_alltypes_pointer.c",
"alltypes.pb.c",
"$COMMON/pb_encode_with_malloc.o",
"$COMMON/pb_common_with_malloc.o",
"$COMMON/malloc_wrappers.o"])
dec = malloc_env.Program(["decode_alltypes_pointer.c",
"alltypes.pb.c",
"$COMMON/pb_decode_with_malloc.o",
"$COMMON/pb_common_with_malloc.o",
"$COMMON/malloc_wrappers.o"])
# Encode and compare results to non-pointer alltypes test case
env.RunTest(enc)
@@ -36,7 +27,7 @@ valgrind = env.WhereIs('valgrind')
kwargs = {}
if valgrind:
kwargs['COMMAND'] = valgrind
kwargs['ARGS'] = ["-q", dec[0].abspath]
kwargs['ARGS'] = ["-q", "--error-exitcode=99", dec[0].abspath]
env.RunTest("decode_alltypes.output", [dec, "encode_alltypes_pointer.output"], **kwargs)

View File

@@ -99,6 +99,8 @@ bool check_alltypes(pb_istream_t *stream, int mode)
TEST(alltypes.opt_bytes == NULL);
TEST(alltypes.opt_submsg == NULL);
TEST(alltypes.opt_enum == NULL);
TEST(alltypes.which_oneof == 0);
}
else
{
@@ -125,6 +127,10 @@ bool check_alltypes(pb_istream_t *stream, int mode)
TEST(alltypes.opt_submsg && *alltypes.opt_submsg->substuff2 == 3056);
TEST(alltypes.opt_enum && *alltypes.opt_enum == MyEnum_Truth);
TEST(alltypes.opt_emptymsg);
TEST(alltypes.which_oneof == AllTypes_oneof_msg1_tag);
TEST(alltypes.oneof.oneof_msg1 && strcmp(alltypes.oneof.oneof_msg1->substuff1, "4059") == 0);
TEST(alltypes.oneof.oneof_msg1->substuff2 && *alltypes.oneof.oneof_msg1->substuff2 == 4059);
}
TEST(alltypes.req_limits->int32_min && *alltypes.req_limits->int32_min == INT32_MIN);

View File

@@ -84,6 +84,9 @@ int main(int argc, char **argv)
MyEnum opt_enum = MyEnum_Truth;
EmptyMessage opt_emptymsg = {0};
static int32_t oneof_substuff = 4059;
SubMessage oneof_msg1 = {"4059", &oneof_substuff};
/* Values for the Limits message. */
static int32_t int32_min = INT32_MIN;
static int32_t int32_max = INT32_MAX;
@@ -164,6 +167,9 @@ int main(int argc, char **argv)
alltypes.opt_submsg = &opt_submsg;
alltypes.opt_enum = &opt_enum;
alltypes.opt_emptymsg = &opt_emptymsg;
alltypes.which_oneof = AllTypes_oneof_msg1_tag;
alltypes.oneof.oneof_msg1 = &oneof_msg1;
}
alltypes.end = &end;

View File

@@ -22,10 +22,7 @@
the decoding and checks the fields. */
bool check_alltypes(pb_istream_t *stream, int mode)
{
AllTypes alltypes;
/* Fill with garbage to better detect initialization errors */
memset(&alltypes, 0xAA, sizeof(alltypes));
AllTypes alltypes = {0};
if (!pb_decode(stream, AllTypes_fields, &alltypes))
return false;

View File

@@ -8,6 +8,7 @@ env.NanopbProto("unittestproto")
# Protocol definitions for basic_buffer/stream tests
env.NanopbProto("person")
#--------------------------------------------
# Binaries of the pb_decode.c and pb_encode.c
# These are built using more strict warning flags.
strict = env.Clone()
@@ -15,3 +16,33 @@ strict.Append(CFLAGS = strict['CORECFLAGS'])
strict.Object("pb_decode.o", "$NANOPB/pb_decode.c")
strict.Object("pb_encode.o", "$NANOPB/pb_encode.c")
strict.Object("pb_common.o", "$NANOPB/pb_common.c")
#-----------------------------------------------
# Binaries of pb_decode etc. with malloc support
# Uses malloc_wrappers.c to count allocations.
malloc_env = env.Clone()
malloc_env.Append(CPPDEFINES = {'PB_ENABLE_MALLOC': 1,
'PB_SYSTEM_HEADER': '\\"malloc_wrappers_syshdr.h\\"'})
malloc_env.Append(CPPPATH = ["$COMMON"])
if 'SYSHDR' in malloc_env:
malloc_env.Append(CPPDEFINES = {'PB_OLD_SYSHDR': malloc_env['SYSHDR']})
# Disable libmudflap, because it will confuse valgrind
# and other memory leak detection tools.
if '-fmudflap' in env["CCFLAGS"]:
malloc_env["CCFLAGS"].remove("-fmudflap")
malloc_env["LINKFLAGS"].remove("-fmudflap")
malloc_env["LIBS"].remove("mudflap")
malloc_strict = malloc_env.Clone()
malloc_strict.Append(CFLAGS = malloc_strict['CORECFLAGS'])
malloc_strict.Object("pb_decode_with_malloc.o", "$NANOPB/pb_decode.c")
malloc_strict.Object("pb_encode_with_malloc.o", "$NANOPB/pb_encode.c")
malloc_strict.Object("pb_common_with_malloc.o", "$NANOPB/pb_common.c")
malloc_env.Object("malloc_wrappers.o", "malloc_wrappers.c")
malloc_env.Depends("$NANOPB/pb.h", ["malloc_wrappers_syshdr.h", "malloc_wrappers.h"])
Export("malloc_env")

View File

@@ -123,16 +123,16 @@ int main()
}
{
pb_istream_t s = S("\x01\xFF\xFF\x03");
pb_istream_t s = S("\x01\x00");
pb_field_t f = {1, PB_LTYPE_VARINT, 0, 0, 4, 0, 0};
uint32_t d;
COMMENT("Test pb_dec_varint using uint32_t")
TEST(pb_dec_varint(&s, &f, &d) && d == 1)
/* Verify that no more than data_size is written. */
d = 0;
d = 0xFFFFFFFF;
f.data_size = 1;
TEST(pb_dec_varint(&s, &f, &d) && (d == 0xFF || d == 0xFF000000))
TEST(pb_dec_varint(&s, &f, &d) && (d == 0xFFFFFF00 || d == 0x00FFFFFF))
}
{

View File

@@ -7,8 +7,8 @@ extend AllTypes {
message ExtensionMessage {
extend AllTypes {
optional ExtensionMessage AllTypes_extensionfield2 = 254;
required ExtensionMessage AllTypes_extensionfield3 = 253;
repeated ExtensionMessage AllTypes_extensionfield4 = 252;
// required ExtensionMessage AllTypes_extensionfield3 = 253; // No longer allowed by protobuf 3
repeated ExtensionMessage AllTypes_extensionfield4 = 252;
}
required string test1 = 1;

View File

@@ -101,6 +101,12 @@ message AllTypes {
optional MyEnum opt_enum = 10057 [default = Second];
optional EmptyMessage opt_emptymsg = 10058;
oneof oneof
{
SubMessage oneof_msg1 = 10059;
EmptyMessage oneof_msg2 = 10060;
}
// Check that extreme integer values are handled correctly
required Limits req_limits = 98;

View File

@@ -101,6 +101,12 @@ message AllTypes {
optional MyEnum opt_enum = 10057 [default = Second];
optional EmptyMessage opt_emptymsg = 10058;
oneof oneof
{
SubMessage oneof_msg1 = 10059;
EmptyMessage oneof_msg2 = 10060;
}
// Check that extreme integer values are handled correctly
required Limits req_limits = 98;

View File

@@ -1,30 +1,9 @@
# Run a fuzz test to verify robustness against corrupted/malicious data.
Import("env")
# We need our own pb_decode.o for the malloc support
env = env.Clone()
env.Append(CPPDEFINES = {'PB_ENABLE_MALLOC': 1,
'PB_SYSTEM_HEADER': '\\"fuzz_syshdr.h\\"'})
env.Append(CPPPATH = ".")
if 'SYSHDR' in env:
env.Append(CPPDEFINES = {'PB_OLD_SYSHDR': env['SYSHDR']})
# Disable libmudflap, because it will confuse valgrind
# and other memory leak detection tools.
if '-fmudflap' in env["CCFLAGS"]:
env["CCFLAGS"].remove("-fmudflap")
env["LINKFLAGS"].remove("-fmudflap")
env["LIBS"].remove("mudflap")
strict = env.Clone()
strict.Append(CFLAGS = strict['CORECFLAGS'])
strict.Object("pb_decode_with_malloc.o", "$NANOPB/pb_decode.c")
strict.Object("pb_encode_with_malloc.o", "$NANOPB/pb_encode.c")
strict.Object("pb_common_with_malloc.o", "$NANOPB/pb_common.c")
Import("env", "malloc_env")
# We want both pointer and static versions of the AllTypes message
# Prefix them with package name.
env.Command("alltypes_static.proto", "#alltypes/alltypes.proto",
lambda target, source, env:
open(str(target[0]), 'w').write("package alltypes_static;\n"
@@ -36,23 +15,22 @@ env.Command("alltypes_pointer.proto", "#alltypes/alltypes.proto",
p1 = env.NanopbProto(["alltypes_pointer", "alltypes_pointer.options"])
p2 = env.NanopbProto(["alltypes_static", "alltypes_static.options"])
fuzz = env.Program(["fuzztest.c",
fuzz = malloc_env.Program(["fuzztest.c",
"alltypes_pointer.pb.c",
"alltypes_static.pb.c",
"pb_encode_with_malloc.o",
"pb_decode_with_malloc.o",
"pb_common_with_malloc.o",
"malloc_wrappers.c"])
Depends([p1, p2, fuzz], ["fuzz_syshdr.h", "malloc_wrappers.h"])
"$COMMON/pb_encode_with_malloc.o",
"$COMMON/pb_decode_with_malloc.o",
"$COMMON/pb_common_with_malloc.o",
"$COMMON/malloc_wrappers.o"])
env.RunTest(fuzz)
fuzzstub = env.Program(["fuzzstub.c",
fuzzstub = malloc_env.Program(["fuzzstub.c",
"alltypes_pointer.pb.c",
"alltypes_static.pb.c",
"pb_encode_with_malloc.o",
"pb_decode_with_malloc.o",
"pb_common_with_malloc.o",
"malloc_wrappers.c"])
"$COMMON/pb_encode_with_malloc.o",
"$COMMON/pb_decode_with_malloc.o",
"$COMMON/pb_common_with_malloc.o",
"$COMMON/malloc_wrappers.o"])

View File

@@ -10,7 +10,7 @@
#include <string.h>
#include <assert.h>
#include <time.h>
#include "malloc_wrappers.h"
#include <malloc_wrappers.h>
#include "alltypes_static.pb.h"
#include "alltypes_pointer.pb.h"

View File

@@ -9,7 +9,7 @@
#include <string.h>
#include <assert.h>
#include <time.h>
#include "malloc_wrappers.h"
#include <malloc_wrappers.h>
#include "alltypes_static.pb.h"
#include "alltypes_pointer.pb.h"

12
tests/intsizes/SConscript Normal file
View File

@@ -0,0 +1,12 @@
# Test that the int_size option in .proto works.
Import('env')
env.NanopbProto('intsizes')
p = env.Program(["intsizes_unittests.c",
"intsizes.pb.c",
"$COMMON/pb_encode.o",
"$COMMON/pb_decode.o",
"$COMMON/pb_common.o"])
env.RunTest(p)

View File

@@ -0,0 +1,39 @@
/* Test the integer size overriding in nanopb options.
* This allows to use 8- and 16-bit integer variables, which are not supported
* directly by Google Protobuf.
*
* The int_size setting will override the number of bits, but keep the type
* otherwise. E.g. uint32 + IS_8 => uint8_t
*/
import 'nanopb.proto';
message IntSizes {
required int32 req_int8 = 1 [(nanopb).int_size = IS_8];
required uint32 req_uint8 = 2 [(nanopb).int_size = IS_8];
required sint32 req_sint8 = 3 [(nanopb).int_size = IS_8];
required int32 req_int16 = 4 [(nanopb).int_size = IS_16];
required uint32 req_uint16 = 5 [(nanopb).int_size = IS_16];
required sint32 req_sint16 = 6 [(nanopb).int_size = IS_16];
required int32 req_int32 = 7 [(nanopb).int_size = IS_32];
required uint32 req_uint32 = 8 [(nanopb).int_size = IS_32];
required sint32 req_sint32 = 9 [(nanopb).int_size = IS_32];
required int32 req_int64 = 10 [(nanopb).int_size = IS_64];
required uint32 req_uint64 = 11 [(nanopb).int_size = IS_64];
required sint32 req_sint64 = 12 [(nanopb).int_size = IS_64];
}
message DefaultSizes {
required int32 req_int8 = 1 ;
required uint32 req_uint8 = 2 ;
required sint32 req_sint8 = 3 ;
required int32 req_int16 = 4 ;
required uint32 req_uint16 = 5 ;
required sint32 req_sint16 = 6 ;
required int32 req_int32 = 7 ;
required uint32 req_uint32 = 8 ;
required sint32 req_sint32 = 9 ;
required int64 req_int64 = 10;
required uint64 req_uint64 = 11;
required sint64 req_sint64 = 12;
}

View File

@@ -0,0 +1,122 @@
#include <stdio.h>
#include <string.h>
#include <pb_decode.h>
#include <pb_encode.h>
#include "unittests.h"
#include "intsizes.pb.h"
#define S(x) pb_istream_from_buffer((uint8_t*)x, sizeof(x) - 1)
/* This is a macro instead of function in order to get the actual values
* into the TEST() lines in output */
#define TEST_ROUNDTRIP(int8, uint8, sint8, \
int16, uint16, sint16, \
int32, uint32, sint32, \
int64, uint64, sint64, expected_result) \
{ \
uint8_t buffer1[128], buffer2[128]; \
size_t msgsize; \
DefaultSizes msg1 = DefaultSizes_init_zero; \
IntSizes msg2 = IntSizes_init_zero; \
\
msg1.req_int8 = int8; \
msg1.req_uint8 = uint8; \
msg1.req_sint8 = sint8; \
msg1.req_int16 = int16; \
msg1.req_uint16 = uint16; \
msg1.req_sint16 = sint16; \
msg1.req_int32 = int32; \
msg1.req_uint32 = uint32; \
msg1.req_sint32 = sint32; \
msg1.req_int64 = int64; \
msg1.req_uint64 = uint64; \
msg1.req_sint64 = sint64; \
\
{ \
pb_ostream_t s = pb_ostream_from_buffer(buffer1, sizeof(buffer1)); \
TEST(pb_encode(&s, DefaultSizes_fields, &msg1)); \
msgsize = s.bytes_written; \
} \
\
{ \
pb_istream_t s = pb_istream_from_buffer(buffer1, msgsize); \
TEST(pb_decode(&s, IntSizes_fields, &msg2) == expected_result); \
if (expected_result) \
{ \
TEST( (int64_t)msg2.req_int8 == int8); \
TEST((uint64_t)msg2.req_uint8 == uint8); \
TEST( (int64_t)msg2.req_sint8 == sint8); \
TEST( (int64_t)msg2.req_int16 == int16); \
TEST((uint64_t)msg2.req_uint16 == uint16); \
TEST( (int64_t)msg2.req_sint16 == sint16); \
TEST( (int64_t)msg2.req_int32 == int32); \
TEST((uint64_t)msg2.req_uint32 == uint32); \
TEST( (int64_t)msg2.req_sint32 == sint32); \
TEST( (int64_t)msg2.req_int64 == int64); \
TEST((uint64_t)msg2.req_uint64 == uint64); \
TEST( (int64_t)msg2.req_sint64 == sint64); \
} \
} \
\
if (expected_result) \
{ \
pb_ostream_t s = pb_ostream_from_buffer(buffer2, sizeof(buffer2)); \
TEST(pb_encode(&s, IntSizes_fields, &msg2)); \
TEST(s.bytes_written == msgsize); \
TEST(memcmp(buffer1, buffer2, msgsize) == 0); \
} \
}
int main()
{
int status = 0;
{
IntSizes msg = IntSizes_init_zero;
COMMENT("Test field sizes");
TEST(sizeof(msg.req_int8) == 1);
TEST(sizeof(msg.req_uint8) == 1);
TEST(sizeof(msg.req_sint8) == 1);
TEST(sizeof(msg.req_int16) == 2);
TEST(sizeof(msg.req_uint16) == 2);
TEST(sizeof(msg.req_sint16) == 2);
TEST(sizeof(msg.req_int32) == 4);
TEST(sizeof(msg.req_uint32) == 4);
TEST(sizeof(msg.req_sint32) == 4);
TEST(sizeof(msg.req_int64) == 8);
TEST(sizeof(msg.req_uint64) == 8);
TEST(sizeof(msg.req_sint64) == 8);
}
COMMENT("Test roundtrip at maximum value");
TEST_ROUNDTRIP(127, 255, 127,
32767, 65535, 32767,
INT32_MAX, UINT32_MAX, INT32_MAX,
INT64_MAX, UINT64_MAX, INT64_MAX, true);
COMMENT("Test roundtrip at minimum value");
TEST_ROUNDTRIP(-128, 0, -128,
-32768, 0, -32768,
INT32_MIN, 0, INT32_MIN,
INT64_MIN, 0, INT64_MIN, true);
COMMENT("Test overflow detection");
TEST_ROUNDTRIP(-129, 0, -128,
-32768, 0, -32768,
INT32_MIN, 0, INT32_MIN,
INT64_MIN, 0, INT64_MIN, false);
TEST_ROUNDTRIP(127, 256, 127,
32767, 65535, 32767,
INT32_MAX, UINT32_MAX, INT32_MAX,
INT64_MAX, UINT64_MAX, INT64_MAX, false);
TEST_ROUNDTRIP(-128, 0, -128,
-32768, 0, -32769,
INT32_MIN, 0, INT32_MIN,
INT64_MIN, 0, INT64_MIN, false);
if (status != 0)
fprintf(stdout, "\n\nSome tests FAILED!\n");
return status;
}

View File

@@ -1,23 +1,6 @@
# Simulate io errors when encoding and decoding
Import("env")
# We need our own pb_decode.o for the malloc support
env = env.Clone()
env.Append(CPPDEFINES = {'PB_ENABLE_MALLOC': 1});
# Disable libmudflap, because it will confuse valgrind
# and other memory leak detection tools.
if '-fmudflap' in env["CCFLAGS"]:
env["CCFLAGS"].remove("-fmudflap")
env["LINKFLAGS"].remove("-fmudflap")
env["LIBS"].remove("mudflap")
strict = env.Clone()
strict.Append(CFLAGS = strict['CORECFLAGS'])
strict.Object("pb_decode_with_malloc.o", "$NANOPB/pb_decode.c")
strict.Object("pb_encode_with_malloc.o", "$NANOPB/pb_encode.c")
strict.Object("pb_common_with_malloc.o", "$NANOPB/pb_common.c")
Import("env", "malloc_env")
c = Copy("$TARGET", "$SOURCE")
env.Command("alltypes.proto", "#alltypes/alltypes.proto", c)
@@ -26,16 +9,17 @@ env.Command("io_errors.c", "#io_errors/io_errors.c", c)
env.NanopbProto(["alltypes", "alltypes.options"])
ioerr = env.Program(["io_errors.c", "alltypes.pb.c",
"pb_encode_with_malloc.o",
"pb_decode_with_malloc.o",
"pb_common_with_malloc.o"])
"$COMMON/pb_encode_with_malloc.o",
"$COMMON/pb_decode_with_malloc.o",
"$COMMON/pb_common_with_malloc.o",
"$COMMON/malloc_wrappers.o"])
# Run tests under valgrind if available
valgrind = env.WhereIs('valgrind')
kwargs = {}
if valgrind:
kwargs['COMMAND'] = valgrind
kwargs['ARGS'] = ["-q", ioerr[0].abspath]
kwargs['ARGS'] = ["-q", "--error-exitcode=99", ioerr[0].abspath]
env.RunTest("io_errors.output", [ioerr, "$BUILD/alltypes/encode_alltypes.output"], **kwargs)

View File

@@ -0,0 +1,13 @@
Import("env", "malloc_env")
env.NanopbProto("mem_release.proto")
test = malloc_env.Program(["mem_release.c",
"mem_release.pb.c",
"$COMMON/pb_encode_with_malloc.o",
"$COMMON/pb_decode_with_malloc.o",
"$COMMON/pb_common_with_malloc.o",
"$COMMON/malloc_wrappers.o"])
env.RunTest(test)

View File

@@ -0,0 +1,185 @@
/* Make sure that all fields are freed in various scenarios. */
#include <pb_decode.h>
#include <pb_encode.h>
#include <malloc_wrappers.h>
#include <stdio.h>
#include <test_helpers.h>
#include "mem_release.pb.h"
#define TEST(x) if (!(x)) { \
fprintf(stderr, "Test " #x " on line %d failed.\n", __LINE__); \
return false; \
}
static char *test_str_arr[] = {"1", "2", ""};
static SubMessage test_msg_arr[] = {SubMessage_init_zero, SubMessage_init_zero};
static pb_extension_t ext1, ext2;
static void fill_TestMessage(TestMessage *msg)
{
msg->static_req_submsg.dynamic_str = "12345";
msg->static_req_submsg.dynamic_str_arr_count = 3;
msg->static_req_submsg.dynamic_str_arr = test_str_arr;
msg->static_req_submsg.dynamic_submsg_count = 2;
msg->static_req_submsg.dynamic_submsg = test_msg_arr;
msg->static_req_submsg.dynamic_submsg[1].dynamic_str = "abc";
msg->static_opt_submsg.dynamic_str = "abc";
msg->has_static_opt_submsg = true;
msg->dynamic_submsg = &msg->static_req_submsg;
msg->extensions = &ext1;
ext1.type = &dynamic_ext;
ext1.dest = &msg->static_req_submsg;
ext1.next = &ext2;
ext2.type = &static_ext;
ext2.dest = &msg->static_req_submsg;
ext2.next = NULL;
}
/* Basic fields, nested submessages, extensions */
static bool test_TestMessage()
{
uint8_t buffer[256];
size_t msgsize;
/* Construct a message with various fields filled in */
{
TestMessage msg = TestMessage_init_zero;
pb_ostream_t stream;
fill_TestMessage(&msg);
stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
if (!pb_encode(&stream, TestMessage_fields, &msg))
{
fprintf(stderr, "Encode failed: %s\n", PB_GET_ERROR(&stream));
return false;
}
msgsize = stream.bytes_written;
}
/* Output encoded message for debug */
SET_BINARY_MODE(stdout);
fwrite(buffer, 1, msgsize, stdout);
/* Decode memory using dynamic allocation */
{
TestMessage msg = TestMessage_init_zero;
pb_istream_t stream;
SubMessage ext2_dest;
msg.extensions = &ext1;
ext1.type = &dynamic_ext;
ext1.dest = NULL;
ext1.next = &ext2;
ext2.type = &static_ext;
ext2.dest = &ext2_dest;
ext2.next = NULL;
stream = pb_istream_from_buffer(buffer, msgsize);
if (!pb_decode(&stream, TestMessage_fields, &msg))
{
fprintf(stderr, "Decode failed: %s\n", PB_GET_ERROR(&stream));
return false;
}
/* Make sure it encodes back to same data */
{
uint8_t buffer2[256];
pb_ostream_t ostream = pb_ostream_from_buffer(buffer2, sizeof(buffer2));
TEST(pb_encode(&ostream, TestMessage_fields, &msg));
TEST(ostream.bytes_written == msgsize);
TEST(memcmp(buffer, buffer2, msgsize) == 0);
}
/* Make sure that malloc counters work */
TEST(get_alloc_count() > 0);
/* Make sure that pb_release releases everything */
pb_release(TestMessage_fields, &msg);
TEST(get_alloc_count() == 0);
/* Check that double-free is a no-op */
pb_release(TestMessage_fields, &msg);
TEST(get_alloc_count() == 0);
}
return true;
}
/* Oneofs */
static bool test_OneofMessage()
{
uint8_t buffer[256];
size_t msgsize;
{
pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
/* Encode first with TestMessage */
{
OneofMessage msg = OneofMessage_init_zero;
msg.which_msgs = OneofMessage_msg1_tag;
fill_TestMessage(&msg.msgs.msg1);
if (!pb_encode(&stream, OneofMessage_fields, &msg))
{
fprintf(stderr, "Encode failed: %s\n", PB_GET_ERROR(&stream));
return false;
}
}
/* Encode second with SubMessage, invoking 'merge' behaviour */
{
OneofMessage msg = OneofMessage_init_zero;
msg.which_msgs = OneofMessage_msg2_tag;
msg.first = 999;
msg.msgs.msg2.dynamic_str = "ABCD";
msg.last = 888;
if (!pb_encode(&stream, OneofMessage_fields, &msg))
{
fprintf(stderr, "Encode failed: %s\n", PB_GET_ERROR(&stream));
return false;
}
}
msgsize = stream.bytes_written;
}
{
OneofMessage msg = OneofMessage_init_zero;
pb_istream_t stream = pb_istream_from_buffer(buffer, msgsize);
if (!pb_decode(&stream, OneofMessage_fields, &msg))
{
fprintf(stderr, "Decode failed: %s\n", PB_GET_ERROR(&stream));
return false;
}
TEST(msg.first == 999);
TEST(msg.which_msgs == OneofMessage_msg2_tag);
TEST(msg.msgs.msg2.dynamic_str);
TEST(strcmp(msg.msgs.msg2.dynamic_str, "ABCD") == 0);
TEST(msg.msgs.msg2.dynamic_str_arr == NULL);
TEST(msg.msgs.msg2.dynamic_submsg == NULL);
TEST(msg.last == 888);
pb_release(OneofMessage_fields, &msg);
TEST(get_alloc_count() == 0);
pb_release(OneofMessage_fields, &msg);
TEST(get_alloc_count() == 0);
}
return true;
}
int main()
{
if (test_TestMessage() && test_OneofMessage())
return 0;
else
return 1;
}

View File

@@ -0,0 +1,34 @@
syntax = "proto2";
import "nanopb.proto";
message SubMessage
{
optional string dynamic_str = 1 [(nanopb).type = FT_POINTER];
repeated string dynamic_str_arr = 2 [(nanopb).type = FT_POINTER];
repeated SubMessage dynamic_submsg = 3 [(nanopb).type = FT_POINTER];
}
message TestMessage
{
required SubMessage static_req_submsg = 1 [(nanopb).type = FT_STATIC];
optional SubMessage dynamic_submsg = 2 [(nanopb).type = FT_POINTER];
optional SubMessage static_opt_submsg = 3 [(nanopb).type = FT_STATIC];
extensions 100 to 200;
}
extend TestMessage
{
optional SubMessage dynamic_ext = 100 [(nanopb).type = FT_POINTER];
optional SubMessage static_ext = 101 [(nanopb).type = FT_STATIC];
}
message OneofMessage
{
required int32 first = 1;
oneof msgs
{
TestMessage msg1 = 2;
SubMessage msg2 = 3;
}
required int32 last = 4;
}

33
tests/oneof/SConscript Normal file
View File

@@ -0,0 +1,33 @@
# Test the 'oneof' feature for generating C unions.
Import('env')
import re
match = None
if 'PROTOC_VERSION' in env:
match = re.search('([0-9]+).([0-9]+).([0-9]+)', env['PROTOC_VERSION'])
if match:
version = map(int, match.groups())
# Oneof is supported by protoc >= 2.6.0
if env.GetOption('clean') or (match and (version[0] > 2 or (version[0] == 2 and version[1] >= 6))):
env.NanopbProto('oneof')
enc = env.Program(['encode_oneof.c',
'oneof.pb.c',
'$COMMON/pb_encode.o',
'$COMMON/pb_common.o'])
dec = env.Program(['decode_oneof.c',
'oneof.pb.c',
'$COMMON/pb_decode.o',
'$COMMON/pb_common.o'])
env.RunTest("message1.pb", enc, ARGS = ['1'])
env.RunTest("message1.txt", [dec, 'message1.pb'], ARGS = ['1'])
env.RunTest("message2.pb", enc, ARGS = ['2'])
env.RunTest("message2.txt", [dec, 'message2.pb'], ARGS = ['2'])
env.RunTest("message3.pb", enc, ARGS = ['3'])
env.RunTest("message3.txt", [dec, 'message3.pb'], ARGS = ['3'])

131
tests/oneof/decode_oneof.c Normal file
View File

@@ -0,0 +1,131 @@
/* Decode a message using oneof fields */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <pb_decode.h>
#include "oneof.pb.h"
#include "test_helpers.h"
#include "unittests.h"
/* Test the 'OneOfMessage' */
int test_oneof_1(pb_istream_t *stream, int option)
{
OneOfMessage msg;
int status = 0;
/* To better catch initialization errors */
memset(&msg, 0xAA, sizeof(msg));
if (!pb_decode(stream, OneOfMessage_fields, &msg))
{
printf("Decoding failed: %s\n", PB_GET_ERROR(stream));
return 1;
}
/* Check that the basic fields work normally */
TEST(msg.prefix == 123);
TEST(msg.suffix == 321);
/* Check that we got the right oneof according to command line */
if (option == 1)
{
TEST(msg.which_values == OneOfMessage_first_tag);
TEST(msg.values.first == 999);
}
else if (option == 2)
{
TEST(msg.which_values == OneOfMessage_second_tag);
TEST(strcmp(msg.values.second, "abcd") == 0);
}
else if (option == 3)
{
TEST(msg.which_values == OneOfMessage_third_tag);
TEST(msg.values.third.array[0] == 1);
TEST(msg.values.third.array[1] == 2);
TEST(msg.values.third.array[2] == 3);
TEST(msg.values.third.array[3] == 4);
TEST(msg.values.third.array[4] == 5);
}
return status;
}
/* Test the 'PlainOneOfMessage' */
int test_oneof_2(pb_istream_t *stream, int option)
{
PlainOneOfMessage msg = PlainOneOfMessage_init_zero;
int status = 0;
if (!pb_decode(stream, PlainOneOfMessage_fields, &msg))
{
printf("Decoding failed: %s\n", PB_GET_ERROR(stream));
return 1;
}
/* Check that we got the right oneof according to command line */
if (option == 1)
{
TEST(msg.which_values == OneOfMessage_first_tag);
TEST(msg.values.first == 999);
}
else if (option == 2)
{
TEST(msg.which_values == OneOfMessage_second_tag);
TEST(strcmp(msg.values.second, "abcd") == 0);
}
else if (option == 3)
{
TEST(msg.which_values == OneOfMessage_third_tag);
TEST(msg.values.third.array[0] == 1);
TEST(msg.values.third.array[1] == 2);
TEST(msg.values.third.array[2] == 3);
TEST(msg.values.third.array[3] == 4);
TEST(msg.values.third.array[4] == 5);
}
return status;
}
int main(int argc, char **argv)
{
uint8_t buffer[OneOfMessage_size];
size_t count;
int option;
if (argc != 2)
{
fprintf(stderr, "Usage: decode_oneof [number]\n");
return 1;
}
option = atoi(argv[1]);
SET_BINARY_MODE(stdin);
count = fread(buffer, 1, sizeof(buffer), stdin);
if (!feof(stdin))
{
printf("Message does not fit in buffer\n");
return 1;
}
{
int status = 0;
pb_istream_t stream;
stream = pb_istream_from_buffer(buffer, count);
status = test_oneof_1(&stream, option);
if (status != 0)
return status;
stream = pb_istream_from_buffer(buffer, count);
status = test_oneof_2(&stream, option);
if (status != 0)
return status;
}
return 0;
}

View File

@@ -0,0 +1,64 @@
/* Encode a message using oneof fields */
#include <stdio.h>
#include <stdlib.h>
#include <pb_encode.h>
#include "oneof.pb.h"
#include "test_helpers.h"
int main(int argc, char **argv)
{
uint8_t buffer[OneOfMessage_size];
OneOfMessage msg = OneOfMessage_init_zero;
pb_ostream_t stream;
int option;
if (argc != 2)
{
fprintf(stderr, "Usage: encode_oneof [number]\n");
return 1;
}
option = atoi(argv[1]);
/* Prefix and suffix are used to test that the union does not disturb
* other fields in the same message. */
msg.prefix = 123;
/* We encode one of the 'values' fields based on command line argument */
if (option == 1)
{
msg.which_values = OneOfMessage_first_tag;
msg.values.first = 999;
}
else if (option == 2)
{
msg.which_values = OneOfMessage_second_tag;
strcpy(msg.values.second, "abcd");
}
else if (option == 3)
{
msg.which_values = OneOfMessage_third_tag;
msg.values.third.array_count = 5;
msg.values.third.array[0] = 1;
msg.values.third.array[1] = 2;
msg.values.third.array[2] = 3;
msg.values.third.array[3] = 4;
msg.values.third.array[4] = 5;
}
msg.suffix = 321;
stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
if (pb_encode(&stream, OneOfMessage_fields, &msg))
{
SET_BINARY_MODE(stdout);
fwrite(buffer, 1, stream.bytes_written, stdout);
return 0;
}
else
{
fprintf(stderr, "Encoding failed: %s\n", PB_GET_ERROR(&stream));
return 1;
}
}

30
tests/oneof/oneof.proto Normal file
View File

@@ -0,0 +1,30 @@
import 'nanopb.proto';
message SubMessage
{
repeated int32 array = 1 [(nanopb).max_count = 8];
}
/* Oneof in a message with other fields */
message OneOfMessage
{
required int32 prefix = 1;
oneof values
{
int32 first = 5;
string second = 6 [(nanopb).max_size = 8];
SubMessage third = 7;
}
required int32 suffix = 99;
}
/* Oneof in a message by itself */
message PlainOneOfMessage
{
oneof values
{
int32 first = 5;
string second = 6 [(nanopb).max_size = 8];
SubMessage third = 7;
}
}

View File

@@ -75,3 +75,12 @@ message SkippedMessage
option (nanopb_msgopt).skip_message = true;
required int32 foo = 1;
}
// Message with oneof field
message OneofMessage
{
oneof foo {
int32 bar = 1;
}
}

View File

@@ -0,0 +1,8 @@
# Regression test for issue 141: wrong encoded size #define for oneof messages
Import("env")
env.NanopbProto("testproto")
env.Object('testproto.pb.c')
env.Match(['testproto.pb.h', 'testproto.expected'])

View File

@@ -0,0 +1,7 @@
define SubMessage_size \s* 88
define OneOfMessage_size \s* 113
define topMessage_size \s* 70
define MyMessage1_size \s* 46
define MyMessage2_size \s* 8
define MyMessage3_size \s* 5
define MyMessage4_size \s* 18

View File

@@ -0,0 +1,50 @@
import 'nanopb.proto';
message SubMessage
{
repeated int32 array = 1 [(nanopb).max_count = 8];
}
message OneOfMessage
{
required int32 prefix = 1;
oneof values
{
int32 first = 5;
string second = 6 [(nanopb).max_size = 8];
SubMessage third = 7;
}
required int32 suffix = 99;
}
message topMessage {
required int32 start = 1;
oneof msg {
MyMessage1 msg1 = 2;
MyMessage2 msg2 = 3;
}
required int32 end = 4;
}
message MyMessage1 {
required uint32 n1 = 1;
required uint32 n2 = 2;
required string s = 3 [(nanopb).max_size = 32];
}
message MyMessage2 {
required uint32 num = 1;
required bool b = 2;
}
message MyMessage3 {
required bool bbb = 1;
required string ss = 2 [(nanopb).max_size = 1];
}
message MyMessage4 {
required bool bbbb = 1;
required string sss = 2 [(nanopb).max_size = 2];
required uint32 num = 3;
required uint32 num2 = 4;
}

View File

@@ -31,7 +31,8 @@ rm $DEST/generator/protoc-gen-nanopb.py
# Package the protoc compiler
cp `which protoc` $DEST/generator-bin/protoc.bin
LIBPROTOC=$(ldd `which protoc` | grep -o '/.*libprotoc[^ ]*')
cp $LIBPROTOC $DEST/generator-bin/
LIBPROTOBUF=$(ldd `which protoc` | grep -o '/.*libprotobuf[^ ]*')
cp $LIBPROTOC $LIBPROTOBUF $DEST/generator-bin/
cat > $DEST/generator-bin/protoc << EOF
#!/bin/bash
SCRIPTDIR=\$(dirname "\$0")