Update changelog for 0.2.8

Publishing nanopb-0.2.8
Fix bug in alltypes test case that made fuzzing difficult.
2014-05-20 19:46:48 +03:00 · 2014-05-20 19:35:00 +03:00 · 2014-05-17 20:28:33 +03:00 · 2014-05-17 20:06:55 +03:00 · 2014-04-26 20:11:54 +03:00 · 2014-04-18 15:40:40 +03:00
39 changed files with 1822 additions and 180 deletions
--- a/CHANGELOG.txt
+++ b/CHANGELOG.txt
@@ -1,3 +1,25 @@
+nanopb-0.2.8 (2014-05-20)
+ Fix security issue with PB_ENABLE_MALLOC. (issue 117)
+ Add option to not add timestamps to .pb.h and .pb.c preambles. (issue 115)
+ Documentation updates
+ Improved tests
+
+nanopb-0.2.7 (2014-04-07)
+ Fix bug with default values for extension fields (issue 111)
+ Fix some MISRA-C warnings (issue 91)
+ Implemented optional malloc() support (issue 80)
+ Changed pointer-type bytes field datatype
+ Add a "found" field to pb_extension_t (issue 112)
+ Add convenience function pb_get_encoded_size() (issue 16)
+
+nanopb-0.2.6 (2014-02-15)
+ Fix generator error with bytes callback fields (issue 99)
+ Fix warnings about large integer constants (issue 102)
+ Add comments to where STATIC_ASSERT is used (issue 96)
+ Add warning about unknown field names on .options (issue 105)
+ Move descriptor.proto to google/protobuf subdirectory (issue 104)
+ Improved tests
+
 nanopb-0.2.5 (2014-01-01)
 Fix a bug with encoding negative values in int32 fields (issue 97)
 Create binary packages of the generator + dependencies (issue 47)
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -47,7 +47,6 @@ Features and limitations

 **Limitations**

-#) User must provide callbacks when decoding arrays or strings without maximum size. Malloc support could be added as a separate module.
 #) Some speed has been sacrificed for code size.
 #) Encoding is focused on writing to streams. For memory buffers only it could be made more efficient.
 #) The deprecated Protocol Buffers feature called "groups" is not supported.
--- a/docs/reference.rst
+++ b/docs/reference.rst
@@ -24,10 +24,8 @@ __BIG_ENDIAN__                 Set this if your platform stores integers and
                               floats in big-endian format. Mixed-endian
                               systems (different layout for ints and floats)
                               are currently not supported.
-NANOPB_INTERNALS               Set this to expose the field encoder functions
-                               that are hidden since nanopb-0.1.3. Starting
-                               with nanopb-0.2.4, this flag does nothing. Use
-                               the newer functions that have better interface.
+PB_ENABLE_MALLOC               Set this to enable dynamic allocation support
+                               in the decoder.
 PB_MAX_REQUIRED_FIELDS         Maximum number of required fields to check for
                               presence. Default value is 64. Increases stack
                               usage 1 byte per every 8 fields. Compiler
@@ -77,8 +75,9 @@ max_count                      Allocated number of entries in arrays
                               (*repeated* fields).
 type                           Type of the generated field. Default value
                               is *FT_DEFAULT*, which selects automatically.
-                               You can use *FT_CALLBACK*, *FT_STATIC* or
-                               *FT_IGNORE* to force a callback field, a static
+                               You can use *FT_CALLBACK*, *FT_POINTER*,
+                               *FT_STATIC* or *FT_IGNORE* to force a callback
+                               field, a dynamically allocated field, a static
                               field or to completely ignore the field.
 long_names                     Prefix the enum name to the enum value in
                               definitions, i.e. *EnumName_EnumValue*. Enabled
@@ -417,6 +416,17 @@ Encodes the contents of a structure as a protocol buffers message and writes it

 Normally pb_encode simply walks through the fields description array and serializes each field in turn. However, submessages must be serialized twice: first to calculate their size and then to actually write them to output. This causes some constraints for callback fields, which must return the same data on every call.

+pb_encode_delimited
+-------------------
+Calculates the length of the message, encodes it as varint and then encodes the message. ::
+
+    bool pb_encode_delimited(pb_ostream_t *stream, const pb_field_t fields[], const void *src_struct);
+
+(parameters are the same as for `pb_encode`_.)
+
+A common way to indicate the message length in Protocol Buffers is to prefix it with a varint.
+This function does this, and it is compatible with *parseDelimitedFrom* in Google's protobuf library.
+
 .. sidebar:: Encoding fields manually

    The functions with names *pb_encode_\** are used when dealing with callback fields. The typical reason for using callbacks is to have an array of unlimited size. In that case, `pb_encode`_ will call your callback function, which in turn will call *pb_encode_\** functions repeatedly to write out values.
@@ -579,6 +589,10 @@ In addition to EOF, the pb_decode implementation supports terminating a message

 For optional fields, this function applies the default value and sets *has_<field>* to false if the field is not present.

+If *PB_ENABLE_MALLOC* is defined, this function may allocate storage for any pointer type fields.
+In this case, you have to call `pb_release`_ to release the memory after you are done with the message.
+On error return `pb_decode` will release the memory itself.
+
 pb_decode_noinit
 ----------------
 Same as `pb_decode`_, except does not apply the default values to fields. ::
@@ -589,6 +603,35 @@ Same as `pb_decode`_, except does not apply the default values to fields. ::

 The destination structure should be filled with zeros before calling this function. Doing a *memset* manually can be slightly faster than using `pb_decode`_ if you don't need any default values.

+In addition to decoding a single message, this function can be used to merge two messages, so that
+values from previous message will remain if the new message does not contain a field.
+
+This function *will not* release the message even on error return. If you use *PB_ENABLE_MALLOC*,
+you will need to call `pb_release`_ yourself.
+
+pb_decode_delimited
+-------------------
+Same as `pb_decode`_, except that it first reads a varint with the length of the message. ::
+
+    bool pb_decode_delimited(pb_istream_t *stream, const pb_field_t fields[], void *dest_struct);
+
+(parameters are the same as for `pb_decode`_.)
+
+A common method to indicate message size in Protocol Buffers is to prefix it with a varint.
+This function is compatible with *writeDelimitedTo* in the Google's Protocol Buffers library.
+
+pb_release
+----------
+Releases any dynamically allocated fields.
+
+    void pb_release(const pb_field_t fields[], void *dest_struct);
+
+:fields:        A field description array. Usually autogenerated.
+:dest_struct:   Pointer to structure where data will be stored.
+
+This function is only available if *PB_ENABLE_MALLOC* is defined. It will release any
+pointer type fields in the structure and set the pointers to NULL.
+
 pb_skip_varint
 --------------
 Skip a varint_ encoded integer without decoding it. ::
--- a/extra/pb_syshdr.h
+++ b/extra/pb_syshdr.h
@@ -53,6 +53,16 @@ typedef int bool;

 #endif

+/* stdlib.h subset */
+#ifdef PB_ENABLE_MALLOC
+#ifdef HAVE_STDLIB_H
+#include <stdlib.h>
+#else
+void *realloc(void *ptr, size_t size);
+void free(void *ptr);
+#endif
+#endif
+
 /* string.h subset */
 #ifdef HAVE_STRING_H
 #include <string.h>
--- a/generator/nanopb_generator.py
+++ b/generator/nanopb_generator.py
@@ -1,7 +1,7 @@
 #!/usr/bin/python

 '''Generate header file for nanopb from a ProtoBuf FileDescriptorSet.'''
-nanopb_version = "nanopb-0.2.5"
+nanopb_version = "nanopb-0.2.8"

 import sys

@@ -15,6 +15,7 @@ except:

 try:
    import google.protobuf.text_format as text_format
+    import google.protobuf.descriptor_pb2 as descriptor
 except:
    sys.stderr.write('''
         *************************************************************
@@ -26,7 +27,7 @@ except:

 try:
    import proto.nanopb_pb2 as nanopb_pb2
-    import proto.descriptor_pb2 as descriptor
+    import proto.plugin_pb2 as plugin_pb2
 except:
    sys.stderr.write('''
         ********************************************************************
@@ -36,7 +37,6 @@ except:
    ''' + '\n')
    raise

-
 # ---------------------------------------------------------------------------
 #                     Generation of single fields
 # ---------------------------------------------------------------------------
@@ -169,6 +169,7 @@ class Field:
        self.max_count = None
        self.array_decl = ""
        self.enc_size = None
+        self.ctype = None
        
        # Parse field options
        if field_options.HasField("max_size"):
@@ -245,7 +246,7 @@ class Field:
                self.ctype = self.struct_name + self.name + 't'
                self.enc_size = varint_max_size(self.max_size) + self.max_size
            elif self.allocation == 'POINTER':
-                self.ctype = 'pb_bytes_ptr_t'
+                self.ctype = 'pb_bytes_array_t'
        elif desc.type == FieldD.TYPE_MESSAGE:
            self.pbtype = 'MESSAGE'
            self.ctype = self.submsgname = names_from_type_name(desc.type_name)
@@ -265,8 +266,8 @@ class Field:
            if self.pbtype == 'MESSAGE':
                # Use struct definition, so recursive submessages are possible
                result += '    struct _%s *%s;' % (self.ctype, self.name)
-            elif self.rules == 'REPEATED' and self.pbtype == 'STRING':
-                # String arrays need to be defined as pointers to pointers
+            elif self.rules == 'REPEATED' and self.pbtype in ['STRING', 'BYTES']:
+                # String/bytes arrays need to be defined as pointers to pointers
                result += '    %s **%s;' % (self.ctype, self.name)
            else:
                result += '    %s *%s;' % (self.ctype, self.name)
@@ -314,6 +315,12 @@ class Field:
            data = self.default.decode('string_escape')
            data = ['0x%02x' % ord(c) for c in data]
            default = '{%d, {%s}}' % (len(data), ','.join(data))
+        elif self.pbtype in ['FIXED32', 'UINT32']:
+            default += 'u'
+        elif self.pbtype in ['FIXED64', 'UINT64']:
+            default += 'ull'
+        elif self.pbtype in ['SFIXED64', 'INT64']:
+            default += 'll'
        
        if declaration_only:
            return 'extern const %s %s_default%s;' % (ctype, self.struct_name + self.name, array_decl)
@@ -344,6 +351,8 @@ class Field:
            result += '0)'
        elif self.pbtype in ['BYTES', 'STRING'] and self.allocation != 'STATIC':
            result += '0)' # Arbitrary size default values not implemented
+        elif self.rules == 'OPTEXT':
+            result += '0)' # Default value for extensions is not implemented
        else:
            result += '&%s_default)' % (self.struct_name + self.name)
        
@@ -698,7 +707,10 @@ def generate_header(dependencies, headername, enums, messages, extensions, optio
    '''
    
    yield '/* Automatically generated nanopb header */\n'
-    yield '/* Generated by %s at %s. */\n\n' % (nanopb_version, time.asctime())
+    if options.notimestamp:
+        yield '/* Generated by %s */\n\n' % (nanopb_version)
+    else:
+        yield '/* Generated by %s at %s. */\n\n' % (nanopb_version, time.asctime())
    
    symbol = make_identifier(headername)
    yield '#ifndef _PB_%s_\n' % symbol
@@ -771,7 +783,10 @@ def generate_source(headername, enums, messages, extensions, options):
    '''Generate content for a source file.'''
    
    yield '/* Automatically generated nanopb constant definitions */\n'
-    yield '/* Generated by %s at %s. */\n\n' % (nanopb_version, time.asctime())
+    if options.notimestamp:
+        yield '/* Generated by %s */\n\n' % (nanopb_version)
+    else:
+        yield '/* Generated by %s at %s. */\n\n' % (nanopb_version, time.asctime())
    yield options.genformat % (headername)
    yield '\n'
    
@@ -815,16 +830,6 @@ def generate_source(headername, enums, messages, extensions, options):
    if worst > 255 or checks:
        yield '\n/* Check that field information fits in pb_field_t */\n'
        
-        if worst < 65536:
-            yield '#if !defined(PB_FIELD_16BIT) && !defined(PB_FIELD_32BIT)\n'
-            if worst > 255:
-                yield '#error Field descriptor for %s is too large. Define PB_FIELD_16BIT to fix this.\n' % worst_field
-            else:
-                assertion = ' && '.join(str(c) + ' < 256' for c in checks)
-                msgs = '_'.join(str(n) for n in checks_msgnames)
-                yield 'STATIC_ASSERT((%s), YOU_MUST_DEFINE_PB_FIELD_16BIT_FOR_MESSAGES_%s)\n'%(assertion,msgs)
-            yield '#endif\n\n'
-        
        if worst > 65535 or checks:
            yield '#if !defined(PB_FIELD_32BIT)\n'
            if worst > 65535:
@@ -832,8 +837,32 @@ def generate_source(headername, enums, messages, extensions, options):
            else:
                assertion = ' && '.join(str(c) + ' < 65536' for c in checks)
                msgs = '_'.join(str(n) for n in checks_msgnames)
+                yield '/* If you get an error here, it means that you need to define PB_FIELD_32BIT\n'
+                yield ' * compile-time option. You can do that in pb.h or on compiler command line.\n'
+                yield ' * \n'
+                yield ' * The reason you need to do this is that some of your messages contain tag\n'
+                yield ' * numbers or field sizes that are larger than what can fit in 8 or 16 bit\n'
+                yield ' * field descriptors.\n'
+                yield ' */\n'
                yield 'STATIC_ASSERT((%s), YOU_MUST_DEFINE_PB_FIELD_32BIT_FOR_MESSAGES_%s)\n'%(assertion,msgs)
-            yield '#endif\n'
+            yield '#endif\n\n'
+        
+        if worst < 65536:
+            yield '#if !defined(PB_FIELD_16BIT) && !defined(PB_FIELD_32BIT)\n'
+            if worst > 255:
+                yield '#error Field descriptor for %s is too large. Define PB_FIELD_16BIT to fix this.\n' % worst_field
+            else:
+                assertion = ' && '.join(str(c) + ' < 256' for c in checks)
+                msgs = '_'.join(str(n) for n in checks_msgnames)
+                yield '/* If you get an error here, it means that you need to define PB_FIELD_16BIT\n'
+                yield ' * compile-time option. You can do that in pb.h or on compiler command line.\n'
+                yield ' * \n'
+                yield ' * The reason you need to do this is that some of your messages contain tag\n'
+                yield ' * numbers or field sizes that are larger than what can fit in the default\n'
+                yield ' * 8 bit descriptors.\n'
+                yield ' */\n'
+                yield 'STATIC_ASSERT((%s), YOU_MUST_DEFINE_PB_FIELD_16BIT_FOR_MESSAGES_%s)\n'%(assertion,msgs)
+            yield '#endif\n\n'
    
    # Add check for sizeof(double)
    has_double = False
@@ -879,6 +908,7 @@ class Globals:
    '''Ugly global variables, should find a good way to pass these.'''
    verbose_options = False
    separate_options = []
+    matched_namemasks = set()

 def get_nanopb_suboptions(subdesc, options, name):
    '''Get copy of options, and merge information from subdesc.'''
@@ -889,6 +919,7 @@ def get_nanopb_suboptions(subdesc, options, name):
    dotname = '.'.join(name.parts)
    for namemask, options in Globals.separate_options:
        if fnmatch(dotname, namemask):
+            Globals.matched_namemasks.add(namemask)
            new_options.MergeFrom(options)
    
    # Handle options defined in .proto
@@ -908,8 +939,8 @@ def get_nanopb_suboptions(subdesc, options, name):
        new_options.MergeFrom(ext)
    
    if Globals.verbose_options:
-        print "Options for " + dotname + ":"
-        print text_format.MessageToString(new_options)
+        sys.stderr.write("Options for " + dotname + ": ")
+        sys.stderr.write(text_format.MessageToString(new_options) + "\n")
    
    return new_options

@@ -938,6 +969,8 @@ optparser.add_option("-Q", "--generated-include-format", dest="genformat",
 optparser.add_option("-L", "--library-include-format", dest="libformat",
    metavar="FORMAT", default='#include <%s>\n',
    help="Set format string to use for including the nanopb pb.h header. [default: %default]")
+optparser.add_option("-T", "--no-timestamp", dest="notimestamp", action="store_true", default=False,
+    help="Don't add timestamp to .pb.h and .pb.c preambles")
 optparser.add_option("-q", "--quiet", dest="quiet", action="store_true", default=False,
    help="Don't print anything except errors.")
 optparser.add_option("-v", "--verbose", dest="verbose", action="store_true", default=False,
@@ -973,13 +1006,14 @@ def process_file(filename, fdesc, options):
        # No %s specified, use the filename as-is
        optfilename = options.options_file
    
-    if options.verbose:
-        print 'Reading options from ' + optfilename
-    
    if os.path.isfile(optfilename):
+        if options.verbose:
+            sys.stderr.write('Reading options from ' + optfilename + '\n')
+
        Globals.separate_options = read_options_file(open(optfilename, "rU"))
    else:
        Globals.separate_options = []
+    Globals.matched_namemasks = set()
    
    # Parse the file
    file_options = get_nanopb_suboptions(fdesc, toplevel_options, Names([filename]))
@@ -1002,6 +1036,14 @@ def process_file(filename, fdesc, options):
    sourcedata = ''.join(generate_source(headerbasename, enums,
                                         messages, extensions, options))

+    # Check if there were any lines in .options that did not match a member
+    unmatched = [n for n,o in Globals.separate_options if n not in Globals.matched_namemasks]
+    if unmatched and not options.quiet:
+        sys.stderr.write("Following patterns in " + optfilename + " did not match any fields: "
+                         + ', '.join(unmatched) + "\n")
+        if not Globals.verbose_options:
+            sys.stderr.write("Use  protoc --nanopb-out=-v:.   to see a list of the field names.\n")
+
    return {'headername': headername, 'headerdata': headerdata,
            'sourcename': sourcename, 'sourcedata': sourcedata}
    
@@ -1023,7 +1065,8 @@ def main_cli():
        results = process_file(filename, None, options)
        
        if not options.quiet:
-            print "Writing to " + results['headername'] + " and " + results['sourcename']
+            sys.stderr.write("Writing to " + results['headername'] + " and "
+                             + results['sourcename'] + "\n")
    
        open(results['headername'], 'w').write(results['headerdata'])
        open(results['sourcename'], 'w').write(results['sourcedata'])        
@@ -1038,7 +1081,6 @@ def main_plugin():
        msvcrt.setmode(sys.stdin.fileno(), os.O_BINARY)
        msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
    
-    import proto.plugin_pb2 as plugin_pb2
    data = sys.stdin.read()
    request = plugin_pb2.CodeGeneratorRequest.FromString(data)
    
@@ -1046,10 +1088,7 @@ def main_plugin():
    args = shlex.split(request.parameter)
    options, dummy = optparser.parse_args(args)
    
-    # We can't go printing stuff to stdout
-    Globals.verbose_options = False
-    options.verbose = False
-    options.quiet = True
+    Globals.verbose_options = options.verbose
    
    response = plugin_pb2.CodeGeneratorResponse()
    
--- a/generator/proto/Makefile
+++ b/generator/proto/Makefile
@@ -1,4 +1,4 @@
-all: nanopb_pb2.py plugin_pb2.py descriptor_pb2.py
+all: nanopb_pb2.py plugin_pb2.py

 %_pb2.py: %.proto
 	protoc --python_out=. $<
--- a/generator/proto/google/protobuf/descriptor.proto
+++ b/generator/proto/google/protobuf/descriptor.proto
--- a/generator/proto/nanopb.proto
+++ b/generator/proto/nanopb.proto
@@ -5,7 +5,7 @@
 // These are used by nanopb to generate statically allocable structures
 // for memory-limited environments.

-import "descriptor.proto";
+import "google/protobuf/descriptor.proto";

 option java_package = "fi.kapsi.koti.jpa.nanopb";

--- a/generator/proto/plugin.proto
+++ b/generator/proto/plugin.proto
@@ -46,7 +46,7 @@

 package google.protobuf.compiler;

-import "descriptor.proto";
+import "google/protobuf/descriptor.proto";

 // An encoded CodeGeneratorRequest is written to the plugin's stdin.
 message CodeGeneratorRequest {
--- a/pb.h
+++ b/pb.h
@@ -10,6 +10,9 @@
 * uncommenting the lines, or on the compiler command line.      *
 *****************************************************************/

+/* Enable support for dynamically allocated fields */
+/* #define PB_ENABLE_MALLOC 1 */
+
 /* Define this if your CPU architecture is big endian, i.e. it
 * stores the most-significant byte first. */
 /* #define __BIG_ENDIAN__ 1 */
@@ -43,7 +46,7 @@

 /* Version of the nanopb library. Just in case you want to check it in
 * your own program. */
-#define NANOPB_VERSION nanopb-0.2.5
+#define NANOPB_VERSION nanopb-0.2.8

 /* Include all the system headers needed by nanopb. You will need the
 * definitions of the following:
@@ -63,6 +66,10 @@
 #include <stddef.h>
 #include <stdbool.h>
 #include <string.h>
+
+#ifdef PB_ENABLE_MALLOC
+#include <stdlib.h>
+#endif
 #endif

 /* Macro for defining packed structures (compiler dependent).
@@ -96,8 +103,14 @@
 #endif

 /* Compile-time assertion, used for checking compatible compilation options.
- * If this fails on your compiler for some reason, use #define STATIC_ASSERT
- * to disable it. */
+ * If this does not work properly on your compiler, use #define STATIC_ASSERT
+ * to disable it.
+ *
+ * But before doing that, check carefully the error message / place where it
+ * comes from to see if the error has a real cause. Unfortunately the error
+ * message is not always very clear to read, but you can see the reason better
+ * in the place where the STATIC_ASSERT macro was called.
+ */
 #ifndef STATIC_ASSERT
 #define STATIC_ASSERT(COND,MSG) typedef char STATIC_ASSERT_MSG(MSG, __LINE__, __COUNTER__)[(COND)?1:-1];
 #define STATIC_ASSERT_MSG(MSG, LINE, COUNTER) STATIC_ASSERT_MSG_(MSG, LINE, COUNTER)
@@ -210,7 +223,11 @@ struct _pb_field_t {
 PB_PACKED_STRUCT_END

 /* Make sure that the standard integer types are of the expected sizes.
- * All kinds of things may break otherwise.. atleast all fixed* types. */
+ * All kinds of things may break otherwise.. atleast all fixed* types.
+ *
+ * If you get errors here, it probably means that your stdint.h is not
+ * correct for your platform.
+ */
 STATIC_ASSERT(sizeof(int8_t) == 1, INT8_T_WRONG_SIZE)
 STATIC_ASSERT(sizeof(uint8_t) == 1, UINT8_T_WRONG_SIZE)
 STATIC_ASSERT(sizeof(int16_t) == 2, INT16_T_WRONG_SIZE)
@@ -224,21 +241,15 @@ STATIC_ASSERT(sizeof(uint64_t) == 8, UINT64_T_WRONG_SIZE)
 * It has the number of bytes in the beginning, and after that an array.
 * Note that actual structs used will have a different length of bytes array.
 */
+#define PB_BYTES_ARRAY_T(n) struct { size_t size; uint8_t bytes[n]; }
+#define PB_BYTES_ARRAY_T_ALLOCSIZE(n) ((size_t)n + offsetof(pb_bytes_array_t, bytes))
+
 struct _pb_bytes_array_t {
    size_t size;
    uint8_t bytes[1];
 };
 typedef struct _pb_bytes_array_t pb_bytes_array_t;

-/* Same, except for pointer-type fields. There is no need to variable struct
- * length in this case.
- */
-struct _pb_bytes_ptr_t {
-    size_t size;
-    uint8_t *bytes;
-};
-typedef struct _pb_bytes_ptr_t pb_bytes_ptr_t;
-
 /* This structure is used for giving the callback function.
 * It is stored in the message structure and filled in by the method that
 * calls pb_decode.
@@ -330,8 +341,23 @@ struct _pb_extension_t {
     * If this extension does not match a field, the next handler is
     * automatically called. */
    pb_extension_t *next;
+
+    /* The decoder sets this to true if the extension was found.
+     * Ignored for encoding. */
+    bool found;
 };

+/* Memory allocation functions to use. You can define pb_realloc and
+ * pb_free to custom functions if you want. */
+#ifdef PB_ENABLE_MALLOC
+#   ifndef pb_realloc
+#       define pb_realloc(ptr, size) realloc(ptr, size)
+#   endif
+#   ifndef pb_free
+#       define pb_free(ptr) free(ptr)
+#   endif
+#endif
+
 /* These macros are used to declare pb_field_t's in the constant array. */
 /* Size of a structure member, in bytes. */
 #define pb_membersize(st, m) (sizeof ((st*)0)->m)
@@ -474,7 +500,11 @@ struct _pb_extension_t {
 * messages if not used.
 */
 #ifdef PB_NO_ERRMSG
-#define PB_RETURN_ERROR(stream,msg) return false
+#define PB_RETURN_ERROR(stream,msg) \
+    do {\
+        UNUSED(stream); \
+        return false; \
+    } while(0)
 #define PB_GET_ERROR(stream) "(errmsg disabled)"
 #else
 #define PB_RETURN_ERROR(stream,msg) \
--- a/pb_decode.c
+++ b/pb_decode.c
@@ -13,7 +13,6 @@
    #define checkreturn __attribute__((warn_unused_result))
 #endif

-#define NANOPB_INTERNALS
 #include "pb.h"
 #include "pb_decode.h"

@@ -130,7 +129,7 @@ bool checkreturn pb_read(pb_istream_t *stream, uint8_t *buf, size_t count)
 * This is an optimization for the varint decoding. */
 static bool checkreturn pb_readbyte(pb_istream_t *stream, uint8_t *buf)
 {
-    if (!stream->bytes_left)
+    if (stream->bytes_left == 0)
        PB_RETURN_ERROR(stream, "end-of-stream");

 #ifndef PB_BUFFER_ONLY
@@ -174,7 +173,7 @@ static bool checkreturn pb_decode_varint32(pb_istream_t *stream, uint32_t *dest)
    if (!pb_readbyte(stream, &byte))
        return false;
    
-    if (!(byte & 0x80))
+    if ((byte & 0x80) == 0)
    {
        /* Quick case, 1 byte value */
        result = byte;
@@ -359,6 +358,10 @@ static bool pb_field_next(pb_field_iterator_t *iter)
    {
        prev_size *= iter->pos->array_size;
    }
+    else if (PB_ATYPE(iter->pos->type) == PB_ATYPE_POINTER)
+    {
+        prev_size = sizeof(void*);
+    }
    
    if (iter->pos->tag == 0)
        return false; /* Only happens with empty message types */
@@ -393,7 +396,7 @@ static bool checkreturn pb_field_find(pb_field_iterator_t *iter, uint32_t tag)
        {
            return true;
        }
-        pb_field_next(iter);
+        (void)pb_field_next(iter);
    } while (iter->field_index != start);
    
    return false;
@@ -431,7 +434,7 @@ static bool checkreturn decode_static_field(pb_istream_t *stream, pb_wire_type_t
                if (!pb_make_string_substream(stream, &substream))
                    return false;
                
-                while (substream.bytes_left && *size < iter->pos->array_size)
+                while (substream.bytes_left > 0 && *size < iter->pos->array_size)
                {
                    void *pItem = (uint8_t*)iter->pData + iter->pos->data_size * (*size);
                    if (!func(&substream, iter->pos, pItem))
@@ -465,6 +468,156 @@ static bool checkreturn decode_static_field(pb_istream_t *stream, pb_wire_type_t
    }
 }

+#ifdef PB_ENABLE_MALLOC
+/* Allocate storage for the field and store the pointer at iter->pData.
+ * array_size is the number of entries to reserve in an array.
+ */
+static bool checkreturn allocate_field(pb_istream_t *stream, void *pData, size_t data_size, size_t array_size)
+{    
+    void *ptr = *(void**)pData;
+    
+    /* Check for multiplication overflows. */
+    size_t size = 0;
+    if (data_size > 0 && array_size > 0)
+    {
+        /* Avoid the costly division if the sizes are small enough.
+         * Multiplication is safe as long as only half of bits are set
+         * in either multiplicand.
+         */
+        const size_t check_limit = (size_t)1 << (sizeof(size_t) * 4);
+        if (data_size >= check_limit || array_size >= check_limit)
+        {
+            if (SIZE_MAX / array_size < data_size)
+            {
+                PB_RETURN_ERROR(stream, "size too large");
+            }
+        }
+    
+        size = array_size * data_size;
+    }
+    
+    /* Allocate new or expand previous allocation */
+    /* Note: on failure the old pointer will remain in the structure,
+     * the message must be freed by caller also on error return. */
+    ptr = pb_realloc(ptr, size);
+    if (ptr == NULL)
+        PB_RETURN_ERROR(stream, "realloc failed");
+    
+    *(void**)pData = ptr;
+    return true;
+}
+
+/* Clear a newly allocated item in case it contains a pointer, or is a submessage. */
+static void initialize_pointer_field(void *pItem, pb_field_iterator_t *iter)
+{
+    if (PB_LTYPE(iter->pos->type) == PB_LTYPE_STRING ||
+        PB_LTYPE(iter->pos->type) == PB_LTYPE_BYTES)
+    {
+        *(void**)pItem = NULL;
+    }
+    else if (PB_LTYPE(iter->pos->type) == PB_LTYPE_SUBMESSAGE)
+    {
+        pb_message_set_to_defaults((const pb_field_t *) iter->pos->ptr, pItem);
+    }
+}
+#endif
+
+static bool checkreturn decode_pointer_field(pb_istream_t *stream, pb_wire_type_t wire_type, pb_field_iterator_t *iter)
+{
+#ifndef PB_ENABLE_MALLOC
+    UNUSED(wire_type);
+    UNUSED(iter);
+    PB_RETURN_ERROR(stream, "no malloc support");
+#else
+    pb_type_t type;
+    pb_decoder_t func;
+    
+    type = iter->pos->type;
+    func = PB_DECODERS[PB_LTYPE(type)];
+    
+    switch (PB_HTYPE(type))
+    {
+        case PB_HTYPE_REQUIRED:
+        case PB_HTYPE_OPTIONAL:
+            if (PB_LTYPE(type) == PB_LTYPE_STRING ||
+                PB_LTYPE(type) == PB_LTYPE_BYTES)
+            {
+                return func(stream, iter->pos, iter->pData);
+            }
+            else
+            {
+                if (!allocate_field(stream, iter->pData, iter->pos->data_size, 1))
+                    return false;
+                
+                initialize_pointer_field(*(void**)iter->pData, iter);
+                return func(stream, iter->pos, *(void**)iter->pData);
+            }
+    
+        case PB_HTYPE_REPEATED:
+            if (wire_type == PB_WT_STRING
+                && PB_LTYPE(type) <= PB_LTYPE_LAST_PACKABLE)
+            {
+                /* Packed array, multiple items come in at once. */
+                bool status = true;
+                size_t *size = (size_t*)iter->pSize;
+                size_t allocated_size = *size;
+                void *pItem;
+                pb_istream_t substream;
+                
+                if (!pb_make_string_substream(stream, &substream))
+                    return false;
+                
+                while (substream.bytes_left)
+                {
+                    if (*size + 1 > allocated_size)
+                    {
+                        /* Allocate more storage. This tries to guess the
+                         * number of remaining entries. Round the division
+                         * upwards. */
+                        allocated_size += (substream.bytes_left - 1) / iter->pos->data_size + 1;
+                        
+                        if (!allocate_field(&substream, iter->pData, iter->pos->data_size, allocated_size))
+                        {
+                            status = false;
+                            break;
+                        }
+                    }
+
+                    /* Decode the array entry */
+                    pItem = *(uint8_t**)iter->pData + iter->pos->data_size * (*size);
+                    initialize_pointer_field(pItem, iter);
+                    if (!func(&substream, iter->pos, pItem))
+                    {
+                        status = false;
+                        break;
+                    }
+                    (*size)++;
+                }
+                pb_close_string_substream(stream, &substream);
+                
+                return status;
+            }
+            else
+            {
+                /* Normal repeated field, i.e. only one item at a time. */
+                size_t *size = (size_t*)iter->pSize;
+                void *pItem;
+                
+                (*size)++;
+                if (!allocate_field(stream, iter->pData, iter->pos->data_size, *size))
+                    return false;
+            
+                pItem = *(uint8_t**)iter->pData + iter->pos->data_size * (*size - 1);
+                initialize_pointer_field(pItem, iter);
+                return func(stream, iter->pos, pItem);
+            }
+            
+        default:
+            PB_RETURN_ERROR(stream, "invalid field type");
+    }
+#endif
+}
+
 static bool checkreturn decode_callback_field(pb_istream_t *stream, pb_wire_type_t wire_type, pb_field_iterator_t *iter)
 {
    pb_callback_t *pCallback = (pb_callback_t*)iter->pData;
@@ -519,6 +672,9 @@ static bool checkreturn decode_field(pb_istream_t *stream, pb_wire_type_t wire_t
        case PB_ATYPE_STATIC:
            return decode_static_field(stream, wire_type, iter);
        
+        case PB_ATYPE_POINTER:
+            return decode_pointer_field(stream, wire_type, iter);
+        
        case PB_ATYPE_CALLBACK:
            return decode_callback_field(stream, wire_type, iter);
        
@@ -534,7 +690,6 @@ static bool checkreturn default_extension_decoder(pb_istream_t *stream,
 {
    const pb_field_t *field = (const pb_field_t*)extension->type->arg;
    pb_field_iterator_t iter;
-    bool dummy;
    
    if (field->tag != tag)
        return true;
@@ -545,7 +700,7 @@ static bool checkreturn default_extension_decoder(pb_istream_t *stream,
    iter.required_field_index = 0;
    iter.dest_struct = extension->dest;
    iter.pData = extension->dest;
-    iter.pSize = &dummy;
+    iter.pSize = &extension->found;
    
    return decode_field(stream, wire_type, &iter);
 }
@@ -558,7 +713,7 @@ static bool checkreturn decode_extension(pb_istream_t *stream,
    pb_extension_t *extension = *(pb_extension_t* const *)iter->pData;
    size_t pos = stream->bytes_left;
    
-    while (extension && pos == stream->bytes_left)
+    while (extension != NULL && pos == stream->bytes_left)
    {
        bool status;
        if (extension->type->decode)
@@ -585,7 +740,7 @@ static bool checkreturn find_extension_field(pb_field_iterator_t *iter)
    do {
        if (PB_LTYPE(iter->pos->type) == PB_LTYPE_EXTENSION)
            return true;
-        pb_field_next(iter);
+        (void)pb_field_next(iter);
    } while (iter->field_index != start);
    
    return false;
@@ -597,45 +752,60 @@ static void pb_message_set_to_defaults(const pb_field_t fields[], void *dest_str
    pb_field_iterator_t iter;
    pb_field_init(&iter, fields, dest_struct);
    
-    /* Initialize size/has fields and apply default values */
    do
    {
        pb_type_t type;
        type = iter.pos->type;
    
+        /* Avoid crash on empty message types (zero fields) */
        if (iter.pos->tag == 0)
            continue;
        
        if (PB_ATYPE(type) == PB_ATYPE_STATIC)
        {
-            /* Initialize the size field for optional/repeated fields to 0. */
            if (PB_HTYPE(type) == PB_HTYPE_OPTIONAL)
            {
+                /* Set has_field to false. Still initialize the optional field
+                 * itself also. */
                *(bool*)iter.pSize = false;
            }
            else if (PB_HTYPE(type) == PB_HTYPE_REPEATED)
            {
+                /* Set array count to 0, no need to initialize contents. */
                *(size_t*)iter.pSize = 0;
-                continue; /* Array is empty, no need to initialize contents */
+                continue;
            }
            
-            /* Initialize field contents to default value */
            if (PB_LTYPE(iter.pos->type) == PB_LTYPE_SUBMESSAGE)
            {
+                /* Initialize submessage to defaults */
                pb_message_set_to_defaults((const pb_field_t *) iter.pos->ptr, iter.pData);
            }
            else if (iter.pos->ptr != NULL)
            {
+                /* Initialize to default value */
                memcpy(iter.pData, iter.pos->ptr, iter.pos->data_size);
            }
            else
            {
+                /* Initialize to zeros */
                memset(iter.pData, 0, iter.pos->data_size);
            }
        }
+        else if (PB_ATYPE(type) == PB_ATYPE_POINTER)
+        {
+            /* Initialize the pointer to NULL. */
+            *(void**)iter.pData = NULL;
+            
+            /* Initialize array count to 0. */
+            if (PB_HTYPE(type) == PB_HTYPE_REPEATED)
+            {
+                *(size_t*)iter.pSize = 0;
+            }
+        }
        else if (PB_ATYPE(type) == PB_ATYPE_CALLBACK)
        {
-            continue; /* Don't overwrite callback */
+            /* Don't overwrite callback */
        }
    } while (pb_field_next(&iter));
 }
@@ -646,7 +816,7 @@ static void pb_message_set_to_defaults(const pb_field_t fields[], void *dest_str

 bool checkreturn pb_decode_noinit(pb_istream_t *stream, const pb_field_t fields[], void *dest_struct)
 {
-    uint8_t fields_seen[(PB_MAX_REQUIRED_FIELDS + 7) / 8] = {0}; /* Used to check for required fields */
+    uint8_t fields_seen[(PB_MAX_REQUIRED_FIELDS + 7) / 8] = {0, 0, 0, 0, 0, 0, 0, 0};
    uint32_t extension_range_start = 0;
    pb_field_iterator_t iter;
    
@@ -722,7 +892,7 @@ bool checkreturn pb_decode_noinit(pb_istream_t *stream, const pb_field_t fields[
        } while (pb_field_next(&iter));
        
        /* Fixup if last field was also required. */
-        if (PB_HTYPE(last_type) == PB_HTYPE_REQUIRED && iter.pos->tag)
+        if (PB_HTYPE(last_type) == PB_HTYPE_REQUIRED && iter.pos->tag != 0)
            req_field_count++;
        
        /* Check the whole bytes */
@@ -742,8 +912,16 @@ bool checkreturn pb_decode_noinit(pb_istream_t *stream, const pb_field_t fields[

 bool checkreturn pb_decode(pb_istream_t *stream, const pb_field_t fields[], void *dest_struct)
 {
+    bool status;
    pb_message_set_to_defaults(fields, dest_struct);
-    return pb_decode_noinit(stream, fields, dest_struct);
+    status = pb_decode_noinit(stream, fields, dest_struct);
+    
+#ifdef PB_ENABLE_MALLOC
+    if (!status)
+        pb_release(fields, dest_struct);
+#endif
+    
+    return status;
 }

 bool pb_decode_delimited(pb_istream_t *stream, const pb_field_t fields[], void *dest_struct)
@@ -759,6 +937,62 @@ bool pb_decode_delimited(pb_istream_t *stream, const pb_field_t fields[], void *
    return status;
 }

+#ifdef PB_ENABLE_MALLOC
+void pb_release(const pb_field_t fields[], void *dest_struct)
+{
+    pb_field_iterator_t iter;
+    pb_field_init(&iter, fields, dest_struct);
+    
+    do
+    {
+        pb_type_t type;
+        type = iter.pos->type;
+    
+        /* Avoid crash on empty message types (zero fields) */
+        if (iter.pos->tag == 0)
+            continue;
+        
+        if (PB_ATYPE(type) == PB_ATYPE_POINTER)
+        {
+            if (PB_HTYPE(type) == PB_HTYPE_REPEATED &&
+                (PB_LTYPE(type) == PB_LTYPE_STRING ||
+                 PB_LTYPE(type) == PB_LTYPE_BYTES))
+            {
+                /* Release entries in repeated string or bytes array */
+                void **pItem = *(void***)iter.pData;
+                size_t count = *(size_t*)iter.pSize;
+                while (count--)
+                {
+                    pb_free(*pItem);
+                    *pItem++ = NULL;
+                }
+            }
+            else if (PB_LTYPE(type) == PB_LTYPE_SUBMESSAGE)
+            {
+                /* Release fields in submessages */
+                void *pItem = *(void**)iter.pData;
+                size_t count = (pItem ? 1 : 0);
+                
+                if (PB_HTYPE(type) == PB_HTYPE_REPEATED)
+                {
+                    count = *(size_t*)iter.pSize;   
+                }
+                
+                while (count--)
+                {
+                    pb_release((const pb_field_t*)iter.pos->ptr, pItem);
+                    pItem = (uint8_t*)pItem + iter.pos->data_size;
+                }
+            }
+            
+            /* Release main item */
+            pb_free(*(void**)iter.pData);
+            *(void**)iter.pData = NULL;
+        }
+    } while (pb_field_next(&iter));
+}
+#endif
+
 /* Field decoders */

 bool pb_decode_svarint(pb_istream_t *stream, int64_t *dest)
@@ -817,7 +1051,7 @@ bool pb_decode_fixed64(pb_istream_t *stream, void *dest)
    #endif   
 }

-bool checkreturn pb_dec_varint(pb_istream_t *stream, const pb_field_t *field, void *dest)
+static bool checkreturn pb_dec_varint(pb_istream_t *stream, const pb_field_t *field, void *dest)
 {
    uint64_t value;
    if (!pb_decode_varint(stream, &value))
@@ -835,7 +1069,7 @@ bool checkreturn pb_dec_varint(pb_istream_t *stream, const pb_field_t *field, vo
    return true;
 }

-bool checkreturn pb_dec_uvarint(pb_istream_t *stream, const pb_field_t *field, void *dest)
+static bool checkreturn pb_dec_uvarint(pb_istream_t *stream, const pb_field_t *field, void *dest)
 {
    uint64_t value;
    if (!pb_decode_varint(stream, &value))
@@ -851,7 +1085,7 @@ bool checkreturn pb_dec_uvarint(pb_istream_t *stream, const pb_field_t *field, v
    return true;
 }

-bool checkreturn pb_dec_svarint(pb_istream_t *stream, const pb_field_t *field, void *dest)
+static bool checkreturn pb_dec_svarint(pb_istream_t *stream, const pb_field_t *field, void *dest)
 {
    int64_t value;
    if (!pb_decode_svarint(stream, &value))
@@ -867,51 +1101,80 @@ bool checkreturn pb_dec_svarint(pb_istream_t *stream, const pb_field_t *field, v
    return true;
 }

-bool checkreturn pb_dec_fixed32(pb_istream_t *stream, const pb_field_t *field, void *dest)
+static bool checkreturn pb_dec_fixed32(pb_istream_t *stream, const pb_field_t *field, void *dest)
 {
    UNUSED(field);
    return pb_decode_fixed32(stream, dest);
 }

-bool checkreturn pb_dec_fixed64(pb_istream_t *stream, const pb_field_t *field, void *dest)
+static bool checkreturn pb_dec_fixed64(pb_istream_t *stream, const pb_field_t *field, void *dest)
 {
    UNUSED(field);
    return pb_decode_fixed64(stream, dest);
 }

-bool checkreturn pb_dec_bytes(pb_istream_t *stream, const pb_field_t *field, void *dest)
-{
-    pb_bytes_array_t *x = (pb_bytes_array_t*)dest;
-    
-    uint32_t temp;
-    if (!pb_decode_varint32(stream, &temp))
-        return false;
-    x->size = temp;
-    
-    /* Check length, noting the space taken by the size_t header. */
-    if (x->size > field->data_size - offsetof(pb_bytes_array_t, bytes))
-        PB_RETURN_ERROR(stream, "bytes overflow");
-    
-    return pb_read(stream, x->bytes, x->size);
-}
-
-bool checkreturn pb_dec_string(pb_istream_t *stream, const pb_field_t *field, void *dest)
+static bool checkreturn pb_dec_bytes(pb_istream_t *stream, const pb_field_t *field, void *dest)
 {
    uint32_t size;
+    pb_bytes_array_t *bdest;
+    
+    if (!pb_decode_varint32(stream, &size))
+        return false;
+    
+    if (PB_ATYPE(field->type) == PB_ATYPE_POINTER)
+    {
+#ifndef PB_ENABLE_MALLOC
+        PB_RETURN_ERROR(stream, "no malloc support");
+#else
+        if (!allocate_field(stream, dest, PB_BYTES_ARRAY_T_ALLOCSIZE(size), 1))
+            return false;
+        bdest = *(pb_bytes_array_t**)dest;
+#endif
+    }
+    else
+    {
+        if (PB_BYTES_ARRAY_T_ALLOCSIZE(size) > field->data_size)
+            PB_RETURN_ERROR(stream, "bytes overflow");
+        bdest = (pb_bytes_array_t*)dest;
+    }
+    
+    bdest->size = size;
+    return pb_read(stream, bdest->bytes, size);
+}
+
+static bool checkreturn pb_dec_string(pb_istream_t *stream, const pb_field_t *field, void *dest)
+{
+    uint32_t size;
+    size_t alloc_size;
    bool status;
    if (!pb_decode_varint32(stream, &size))
        return false;
    
-    /* Check length, noting the null terminator */
-    if (size + 1 > field->data_size)
-        PB_RETURN_ERROR(stream, "string overflow");
+    /* Space for null terminator */
+    alloc_size = size + 1;
+    
+    if (PB_ATYPE(field->type) == PB_ATYPE_POINTER)
+    {
+#ifndef PB_ENABLE_MALLOC
+        PB_RETURN_ERROR(stream, "no malloc support");
+#else
+        if (!allocate_field(stream, dest, alloc_size, 1))
+            return false;
+        dest = *(void**)dest;
+#endif
+    }
+    else
+    {
+        if (alloc_size > field->data_size)
+            PB_RETURN_ERROR(stream, "string overflow");
+    }
    
    status = pb_read(stream, (uint8_t*)dest, size);
    *((uint8_t*)dest + size) = 0;
    return status;
 }

-bool checkreturn pb_dec_submessage(pb_istream_t *stream, const pb_field_t *field, void *dest)
+static bool checkreturn pb_dec_submessage(pb_istream_t *stream, const pb_field_t *field, void *dest)
 {
    bool status;
    pb_istream_t substream;
--- a/pb_decode.h
+++ b/pb_decode.h
@@ -73,6 +73,9 @@ bool pb_decode(pb_istream_t *stream, const pb_field_t fields[], void *dest_struc
 *
 * This can also be used for 'merging' two messages, i.e. update only the
 * fields that exist in the new message.
+ *
+ * Note: If this function returns with an error, it will not release any
+ * dynamically allocated fields. You will need to call pb_release() yourself.
 */
 bool pb_decode_noinit(pb_istream_t *stream, const pb_field_t fields[], void *dest_struct);

@@ -82,6 +85,14 @@ bool pb_decode_noinit(pb_istream_t *stream, const pb_field_t fields[], void *des
 */
 bool pb_decode_delimited(pb_istream_t *stream, const pb_field_t fields[], void *dest_struct);

+#ifdef PB_ENABLE_MALLOC
+/* Release any allocated pointer fields. If you use dynamic allocation, you should
+ * call this for any successfully decoded message when you are done with it. If
+ * pb_decode() returns with an error, the message is already released.
+ */
+void pb_release(const pb_field_t fields[], void *dest_struct);
+#endif
+

 /**************************************
 * Functions for manipulating streams *
--- a/pb_encode.c
+++ b/pb_encode.c
@@ -3,7 +3,6 @@
 * 2011 Petteri Aimonen <jpa@kapsi.fi>
 */

-#define NANOPB_INTERNALS
 #include "pb.h"
 #include "pb_encode.h"

@@ -174,11 +173,12 @@ static bool checkreturn encode_array(pb_ostream_t *stream, const pb_field_t *fie
                return false;

            /* Normally the data is stored directly in the array entries, but
-             * for pointer-type string fields, the array entries are actually
-             * string pointers. So we have to dereference once more to get to
-             * the character data. */
+             * for pointer-type string and bytes fields, the array entries are
+             * actually pointers themselves also. So we have to dereference once
+             * more to get to the actual data. */
            if (PB_ATYPE(field->type) == PB_ATYPE_POINTER &&
-                PB_LTYPE(field->type) == PB_LTYPE_STRING)
+                (PB_LTYPE(field->type) == PB_LTYPE_STRING ||
+                 PB_LTYPE(field->type) == PB_LTYPE_BYTES))
            {
                if (!func(stream, field, *(const void* const*)p))
                    return false;      
@@ -378,6 +378,17 @@ bool pb_encode_delimited(pb_ostream_t *stream, const pb_field_t fields[], const
    return pb_encode_submessage(stream, fields, src_struct);
 }

+bool pb_get_encoded_size(size_t *size, const pb_field_t fields[], const void *src_struct)
+{
+    pb_ostream_t stream = PB_OSTREAM_SIZING;
+    
+    if (!pb_encode(&stream, fields, src_struct))
+        return false;
+    
+    *size = stream.bytes_written;
+    return true;
+}
+
 /********************
 * Helper functions *
 ********************/
@@ -404,9 +415,9 @@ bool checkreturn pb_encode_svarint(pb_ostream_t *stream, int64_t value)
 {
    uint64_t zigzagged;
    if (value < 0)
-        zigzagged = (uint64_t)(~(value << 1));
+        zigzagged = ~((uint64_t)value << 1);
    else
-        zigzagged = (uint64_t)(value << 1);
+        zigzagged = (uint64_t)value << 1;
    
    return pb_encode_varint(stream, zigzagged);
 }
@@ -447,7 +458,7 @@ bool checkreturn pb_encode_fixed64(pb_ostream_t *stream, const void *value)

 bool checkreturn pb_encode_tag(pb_ostream_t *stream, pb_wire_type_t wiretype, uint32_t field_number)
 {
-    uint64_t tag = wiretype | (field_number << 3);
+    uint64_t tag = ((uint64_t)field_number << 3) | wiretype;
    return pb_encode_varint(stream, tag);
 }

@@ -543,7 +554,7 @@ bool checkreturn pb_encode_submessage(pb_ostream_t *stream, const pb_field_t fie

 /* Field encoders */

-bool checkreturn pb_enc_varint(pb_ostream_t *stream, const pb_field_t *field, const void *src)
+static bool checkreturn pb_enc_varint(pb_ostream_t *stream, const pb_field_t *field, const void *src)
 {
    int64_t value = 0;
    
@@ -561,7 +572,7 @@ bool checkreturn pb_enc_varint(pb_ostream_t *stream, const pb_field_t *field, co
    return pb_encode_varint(stream, (uint64_t)value);
 }

-bool checkreturn pb_enc_uvarint(pb_ostream_t *stream, const pb_field_t *field, const void *src)
+static bool checkreturn pb_enc_uvarint(pb_ostream_t *stream, const pb_field_t *field, const void *src)
 {
    uint64_t value = 0;
    
@@ -575,7 +586,7 @@ bool checkreturn pb_enc_uvarint(pb_ostream_t *stream, const pb_field_t *field, c
    return pb_encode_varint(stream, value);
 }

-bool checkreturn pb_enc_svarint(pb_ostream_t *stream, const pb_field_t *field, const void *src)
+static bool checkreturn pb_enc_svarint(pb_ostream_t *stream, const pb_field_t *field, const void *src)
 {
    int64_t value = 0;
    
@@ -589,36 +600,38 @@ bool checkreturn pb_enc_svarint(pb_ostream_t *stream, const pb_field_t *field, c
    return pb_encode_svarint(stream, value);
 }

-bool checkreturn pb_enc_fixed64(pb_ostream_t *stream, const pb_field_t *field, const void *src)
+static bool checkreturn pb_enc_fixed64(pb_ostream_t *stream, const pb_field_t *field, const void *src)
 {
    UNUSED(field);
    return pb_encode_fixed64(stream, src);
 }

-bool checkreturn pb_enc_fixed32(pb_ostream_t *stream, const pb_field_t *field, const void *src)
+static bool checkreturn pb_enc_fixed32(pb_ostream_t *stream, const pb_field_t *field, const void *src)
 {
    UNUSED(field);
    return pb_encode_fixed32(stream, src);
 }

-bool checkreturn pb_enc_bytes(pb_ostream_t *stream, const pb_field_t *field, const void *src)
+static bool checkreturn pb_enc_bytes(pb_ostream_t *stream, const pb_field_t *field, const void *src)
 {
-    if (PB_ATYPE(field->type) == PB_ATYPE_POINTER)
+    const pb_bytes_array_t *bytes = (const pb_bytes_array_t*)src;
+    
+    if (src == NULL)
    {
-        const pb_bytes_ptr_t *bytes = (const pb_bytes_ptr_t*)src;
-        return pb_encode_string(stream, bytes->bytes, bytes->size);
+        /* Threat null pointer as an empty bytes field */
+        return pb_encode_string(stream, NULL, 0);
    }
-    else
+    
+    if (PB_ATYPE(field->type) == PB_ATYPE_STATIC &&
+        PB_BYTES_ARRAY_T_ALLOCSIZE(bytes->size) > field->data_size)
    {
-        const pb_bytes_array_t *bytes = (const pb_bytes_array_t*)src;
-        if (bytes->size + offsetof(pb_bytes_array_t, bytes) > field->data_size)
-            PB_RETURN_ERROR(stream, "bytes size exceeded");
-
-        return pb_encode_string(stream, bytes->bytes, bytes->size);
+        PB_RETURN_ERROR(stream, "bytes size exceeded");
    }
+    
+    return pb_encode_string(stream, bytes->bytes, bytes->size);
 }

-bool checkreturn pb_enc_string(pb_ostream_t *stream, const pb_field_t *field, const void *src)
+static bool checkreturn pb_enc_string(pb_ostream_t *stream, const pb_field_t *field, const void *src)
 {
    /* strnlen() is not always available, so just use a loop */
    size_t size = 0;
@@ -628,16 +641,23 @@ bool checkreturn pb_enc_string(pb_ostream_t *stream, const pb_field_t *field, co
    if (PB_ATYPE(field->type) == PB_ATYPE_POINTER)
        max_size = (size_t)-1;

-    while (size < max_size && *p != '\0')
+    if (src == NULL)
    {
-        size++;
-        p++;
+        size = 0; /* Threat null pointer as an empty string */
+    }
+    else
+    {
+        while (size < max_size && *p != '\0')
+        {
+            size++;
+            p++;
+        }
    }

    return pb_encode_string(stream, (const uint8_t*)src, size);
 }

-bool checkreturn pb_enc_submessage(pb_ostream_t *stream, const pb_field_t *field, const void *src)
+static bool checkreturn pb_enc_submessage(pb_ostream_t *stream, const pb_field_t *field, const void *src)
 {
    if (field->ptr == NULL)
        PB_RETURN_ERROR(stream, "invalid field descriptor");
--- a/pb_encode.h
+++ b/pb_encode.h
@@ -71,6 +71,10 @@ bool pb_encode(pb_ostream_t *stream, const pb_field_t fields[], const void *src_
 */
 bool pb_encode_delimited(pb_ostream_t *stream, const pb_field_t fields[], const void *src_struct);

+/* Encode the message to get the size of the encoded data, but do not store
+ * the data. */
+bool pb_get_encoded_size(size_t *size, const pb_field_t fields[], const void *src_struct);
+
 /**************************************
 * Functions for manipulating streams *
 **************************************/
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -4,3 +4,18 @@ all:
 clean:
 	scons -c

+coverage:
+	rm -rf build coverage
+
+	# LCOV does not like the newer gcov format
+	scons CC=gcc-4.6 CXX=gcc-4.6
+
+	# We are only interested in pb_encode.o and pb_decode.o
+	find build -name '*.gcda' -and \! \( -name '*pb_encode*' -or -name '*pb_decode*' \) -exec rm '{}' \;
+
+	# Collect the data
+	mkdir build/coverage
+	lcov --base-directory . --directory build/ --gcov-tool gcov-4.6 -c -o build/coverage/nanopb.info
+
+	# Generate HTML
+	genhtml -o build/coverage build/coverage/nanopb.info
--- a/tests/SConstruct
+++ b/tests/SConstruct
@@ -19,8 +19,8 @@ env = Environment(ENV = os.environ, tools = ['default', 'nanopb'])
 # Allow overriding the compiler with scons CC=???
 if 'CC' in ARGUMENTS: env.Replace(CC = ARGUMENTS['CC'])
 if 'CXX' in ARGUMENTS: env.Replace(CXX = ARGUMENTS['CXX'])
-if 'CFLAGS' in ARGUMENTS: env.Append(CCFLAGS = ARGUMENTS['CFLAGS'])
-if 'CXXFLAGS' in ARGUMENTS: env.Append(CCFLAGS = ARGUMENTS['CXXFLAGS'])
+if 'CCFLAGS' in ARGUMENTS: env.Append(CCFLAGS = ARGUMENTS['CCFLAGS'])
+if 'CXXFLAGS' in ARGUMENTS: env.Append(CXXFLAGS = ARGUMENTS['CXXFLAGS'])

 # Add the builders defined in site_init.py
 add_nanopb_builders(env)
@@ -33,13 +33,16 @@ env.Append(PROTOCPATH = '#../generator')

 # Check the compilation environment, unless we are just cleaning up.
 if not env.GetOption('clean'):
-    def check_ccflags(context, flags):
+    def check_ccflags(context, flags, linkflags = ''):
        '''Check if given CCFLAGS are supported'''
        context.Message('Checking support for CCFLAGS="%s"... ' % flags)
        oldflags = context.env['CCFLAGS']
+        oldlinkflags = context.env['CCFLAGS']
        context.env.Append(CCFLAGS = flags)
+        context.env.Append(LINKFLAGS = linkflags)
        result = context.TryCompile("int main() {return 0;}", '.c')
        context.env.Replace(CCFLAGS = oldflags)
+        context.env.Replace(LINKFLAGS = oldlinkflags)
        context.Result(result)
        return result
    
@@ -50,6 +53,7 @@ if not env.GetOption('clean'):
    stdint = conf.CheckCHeader('stdint.h')
    stddef = conf.CheckCHeader('stddef.h')
    string = conf.CheckCHeader('string.h')
+    stdlib = conf.CheckCHeader('stdlib.h')
    if not stdbool or not stdint or not stddef or not string:
        conf.env.Append(CPPDEFINES = {'PB_SYSTEM_HEADER': '\\"pb_syshdr.h\\"'})
        conf.env.Append(CPPPATH = "#../extra")
@@ -58,7 +62,8 @@ if not env.GetOption('clean'):
        if stdint: conf.env.Append(CPPDEFINES = {'HAVE_STDINT_H': 1})
        if stddef: conf.env.Append(CPPDEFINES = {'HAVE_STDDEF_H': 1})
        if string: conf.env.Append(CPPDEFINES = {'HAVE_STRING_H': 1})
-        
+        if stdlib: conf.env.Append(CPPDEFINES = {'HAVE_STDLIB_H': 1})
+    
    # Check if we can use pkg-config to find protobuf include path
    status, output = conf.TryAction('pkg-config protobuf --variable=includedir > $TARGET')
    if status:
@@ -70,16 +75,24 @@ if not env.GetOption('clean'):
    if 'gcc' in env['CC']:
        if conf.CheckLib('mudflap'):
            conf.env.Append(CCFLAGS = '-fmudflap')
-            conf.env.Append(LINKFLAGS = '-lmudflap -fmudflap')
+            conf.env.Append(LINKFLAGS = '-fmudflap')
    
    # Check if we can use extra strict warning flags (only with GCC)
    extra = '-Wcast-qual -Wlogical-op -Wconversion'
+    extra += ' -fstrict-aliasing -Wstrict-aliasing=1'
    extra += ' -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls'
    extra += ' -Wstack-protector '
    if 'gcc' in env['CC']:
        if conf.CheckCCFLAGS(extra):
            conf.env.Append(CORECFLAGS = extra)
    
+    # Check if we can use undefined behaviour sanitizer (only with clang)
+    extra = '-fsanitize=undefined '
+    if 'clang' in env['CC']:
+        if conf.CheckCCFLAGS(extra, linkflags = extra):
+            conf.env.Append(CORECFLAGS = extra)
+            conf.env.Append(LINKFLAGS = extra)
+    
    # End the config stuff
    env = conf.Finish()

@@ -88,15 +101,15 @@ if 'gcc' in env['CC']:
    # GNU Compiler Collection
    
    # Debug info, warnings as errors
-    env.Append(CFLAGS = '-ansi -pedantic -g -O0 -Wall -Werror --coverage -fstack-protector-all')
+    env.Append(CFLAGS = '-ansi -pedantic -g -Wall -Werror -fprofile-arcs -ftest-coverage -fstack-protector-all')
    env.Append(CORECFLAGS = '-Wextra')
-    env.Append(LINKFLAGS = '--coverage')
+    env.Append(LINKFLAGS = '-g --coverage')
    
    # We currently need uint64_t anyway, even though ANSI C90 otherwise..
    env.Append(CFLAGS = '-Wno-long-long')
 elif 'clang' in env['CC']:
    # CLang
-    env.Append(CFLAGS = '-ansi -g -O0 -Wall -Werror')
+    env.Append(CFLAGS = '-ansi -g -Wall -Werror')
    env.Append(CORECFLAGS = ' -Wextra -Wcast-qual -Wconversion')
 elif 'cl' in env['CC']:
    # Microsoft Visual C++
@@ -116,10 +129,10 @@ elif 'tcc' in env['CC']:

 env.SetDefault(CORECFLAGS = '')

-if 'clang++' in env['CXX']:
-    env.Append(CXXFLAGS = '-g -O0 -Wall -Werror -Wextra -Wno-missing-field-initializers')
-elif 'g++' in env['CXX']:
-    env.Append(CXXFLAGS = '-g -O0 -Wall -Werror -Wextra -Wno-missing-field-initializers')
+if 'clang' in env['CXX']:
+    env.Append(CXXFLAGS = '-g -Wall -Werror -Wextra -Wno-missing-field-initializers')
+elif 'g++' in env['CXX'] or 'gcc' in env['CXX']:
+    env.Append(CXXFLAGS = '-g -Wall -Werror -Wextra -Wno-missing-field-initializers')
 elif 'cl' in env['CXX']:
    env.Append(CXXFLAGS = '/Zi /W2 /WX')
    
--- a/tests/alltypes/alltypes.proto
+++ b/tests/alltypes/alltypes.proto
@@ -14,16 +14,16 @@ enum HugeEnum {
 }

 message Limits {
-    required int32      int32_min  =  1;
-    required int32      int32_max  =  2;
-    required uint32     uint32_min =  3;
-    required uint32     uint32_max =  4;
-    required int64      int64_min  =  5;
-    required int64      int64_max  =  6;
-    required uint64     uint64_min =  7;
-    required uint64     uint64_max =  8;
-    required HugeEnum   enum_min   =  9;
-    required HugeEnum   enum_max   = 10;
+    required int32      int32_min  =  1 [default = 2147483647];
+    required int32      int32_max  =  2 [default = -2147483647];
+    required uint32     uint32_min =  3 [default = 4294967295];
+    required uint32     uint32_max =  4 [default = 0];
+    required int64      int64_min  =  5 [default = 9223372036854775807];
+    required int64      int64_max  =  6 [default = -9223372036854775807];
+    required uint64     uint64_min =  7 [default = 18446744073709551615];
+    required uint64     uint64_max =  8 [default = 0];
+    required HugeEnum   enum_min   =  9 [default = Positive];
+    required HugeEnum   enum_max   = 10 [default = Negative];
 }

 enum MyEnum {
--- a/tests/alltypes/decode_alltypes.c
+++ b/tests/alltypes/decode_alltypes.c
@@ -23,6 +23,7 @@ bool check_alltypes(pb_istream_t *stream, int mode)
    
    /* Fill with garbage to better detect initialization errors */
    memset(&alltypes, 0xAA, sizeof(alltypes));
+    alltypes.extensions = 0;
    
    if (!pb_decode(stream, AllTypes_fields, &alltypes))
        return false;
--- a/tests/alltypes_callback/SConscript
+++ b/tests/alltypes_callback/SConscript
@@ -0,0 +1,23 @@
+# Test the AllTypes encoding & decoding using callbacks for all fields.
+
+Import("env")
+
+c = Copy("$TARGET", "$SOURCE")
+env.Command("alltypes.proto", "#alltypes/alltypes.proto", c)
+
+env.NanopbProto(["alltypes", "alltypes.options"])
+enc = env.Program(["encode_alltypes_callback.c", "alltypes.pb.c", "$COMMON/pb_encode.o"])
+dec = env.Program(["decode_alltypes_callback.c", "alltypes.pb.c", "$COMMON/pb_decode.o"])
+
+refdec = "$BUILD/alltypes/decode_alltypes$PROGSUFFIX"
+
+# Encode and compare results
+env.RunTest(enc)
+env.RunTest("decode_alltypes.output", [refdec, "encode_alltypes_callback.output"])
+env.RunTest("decode_alltypes_callback.output", [dec, "encode_alltypes_callback.output"])
+
+# Do the same thing with the optional fields present
+env.RunTest("optionals.output", enc, ARGS = ['1'])
+env.RunTest("optionals.refdecout", [refdec, "optionals.output"], ARGS = ['1'])
+env.RunTest("optionals.decout", [dec, "optionals.output"], ARGS = ['1'])
+
--- a/tests/alltypes_callback/alltypes.options
+++ b/tests/alltypes_callback/alltypes.options
@@ -0,0 +1,3 @@
+# Generate all fields as callbacks.
+AllTypes.*              type:FT_CALLBACK
+SubMessage.substuff1    max_size:16
--- a/tests/alltypes_callback/decode_alltypes_callback.c
+++ b/tests/alltypes_callback/decode_alltypes_callback.c
@@ -0,0 +1,424 @@
+/* Attempts to test all the datatypes supported by ProtoBuf when used as callback fields.
+ * Note that normally there would be no reason to use callback fields for this,
+ * because each encoder defined here only gives a single field.
+ */
+
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <pb_decode.h>
+#include "alltypes.pb.h"
+#include "test_helpers.h"
+
+#define TEST(x) if (!(x)) { \
+    printf("Test " #x " failed (in field %d).\n", field->tag); \
+    return false; \
+    }
+
+static bool read_varint(pb_istream_t *stream, const pb_field_t *field, void **arg)
+{
+    uint64_t value;
+    if (!pb_decode_varint(stream, &value))
+        return false;
+    
+    TEST((int64_t)value == (long)*arg);
+    return true;
+}
+
+static bool read_svarint(pb_istream_t *stream, const pb_field_t *field, void **arg)
+{
+    int64_t value;
+    if (!pb_decode_svarint(stream, &value))
+        return false;
+    
+    TEST(value == (long)*arg);
+    return true;
+}
+
+static bool read_fixed32(pb_istream_t *stream, const pb_field_t *field, void **arg)
+{
+    uint32_t value;
+    if (!pb_decode_fixed32(stream, &value))
+        return false;
+    
+    TEST(value == *(uint32_t*)*arg);
+    return true;
+}
+
+static bool read_fixed64(pb_istream_t *stream, const pb_field_t *field, void **arg)
+{
+    uint64_t value;
+    if (!pb_decode_fixed64(stream, &value))
+        return false;
+    
+    TEST(value == *(uint64_t*)*arg);
+    return true;
+}
+
+static bool read_string(pb_istream_t *stream, const pb_field_t *field, void **arg)
+{
+    uint8_t buf[16] = {0};
+    size_t len = stream->bytes_left;
+    
+    if (len > sizeof(buf) - 1 || !pb_read(stream, buf, len))
+        return false;
+    
+    TEST(strcmp((char*)buf, *arg) == 0);
+    return true;
+}
+
+static bool read_submsg(pb_istream_t *stream, const pb_field_t *field, void **arg)
+{
+    SubMessage submsg = {""};
+    
+    if (!pb_decode(stream, SubMessage_fields, &submsg))
+        return false;
+    
+    TEST(memcmp(&submsg, *arg, sizeof(submsg)));
+    return true;
+}
+
+static bool read_emptymsg(pb_istream_t *stream, const pb_field_t *field, void **arg)
+{
+    EmptyMessage emptymsg = {0};
+    return pb_decode(stream, EmptyMessage_fields, &emptymsg);
+}
+
+static bool read_repeated_varint(pb_istream_t *stream, const pb_field_t *field, void **arg)
+{
+    int32_t** expected = (int32_t**)arg;
+    uint64_t value;
+    if (!pb_decode_varint(stream, &value))
+        return false;
+
+    TEST(*(*expected)++ == value);
+    return true;
+}
+
+static bool read_repeated_svarint(pb_istream_t *stream, const pb_field_t *field, void **arg)
+{
+    int32_t** expected = (int32_t**)arg;
+    int64_t value;
+    if (!pb_decode_svarint(stream, &value))
+        return false;
+
+    TEST(*(*expected)++ == value);
+    return true;
+}
+
+static bool read_repeated_fixed32(pb_istream_t *stream, const pb_field_t *field, void **arg)
+{
+    uint32_t** expected = (uint32_t**)arg;
+    uint32_t value;
+    if (!pb_decode_fixed32(stream, &value))
+        return false;
+
+    TEST(*(*expected)++ == value);
+    return true;
+}
+
+static bool read_repeated_fixed64(pb_istream_t *stream, const pb_field_t *field, void **arg)
+{
+    uint64_t** expected = (uint64_t**)arg;
+    uint64_t value;
+    if (!pb_decode_fixed64(stream, &value))
+        return false;
+
+    TEST(*(*expected)++ == value);
+    return true;
+}
+
+static bool read_repeated_string(pb_istream_t *stream, const pb_field_t *field, void **arg)
+{
+    uint8_t*** expected = (uint8_t***)arg;
+    uint8_t buf[16] = {0};
+    size_t len = stream->bytes_left;
+    
+    if (len > sizeof(buf) - 1 || !pb_read(stream, buf, len))
+        return false;
+    
+    TEST(strcmp((char*)*(*expected)++, (char*)buf) == 0);
+    return true;
+}
+
+static bool read_repeated_submsg(pb_istream_t *stream, const pb_field_t *field, void **arg)
+{
+    SubMessage** expected = (SubMessage**)arg;
+    SubMessage decoded = {""};
+    if (!pb_decode(stream, SubMessage_fields, &decoded))
+        return false;
+
+    TEST(memcmp((*expected)++, &decoded, sizeof(decoded)) == 0);
+    return true;
+}
+
+static bool read_limits(pb_istream_t *stream, const pb_field_t *field, void **arg)
+{
+    Limits decoded = {0};
+    if (!pb_decode(stream, Limits_fields, &decoded))
+        return false;
+
+    TEST(decoded.int32_min  == INT32_MIN);
+    TEST(decoded.int32_max  == INT32_MAX);
+    TEST(decoded.uint32_min == 0);
+    TEST(decoded.uint32_max == UINT32_MAX);
+    TEST(decoded.int64_min  == INT64_MIN);
+    TEST(decoded.int64_max  == INT64_MAX);
+    TEST(decoded.uint64_min == 0);
+    TEST(decoded.uint64_max == UINT64_MAX);
+    TEST(decoded.enum_min   == HugeEnum_Negative);
+    TEST(decoded.enum_max   == HugeEnum_Positive);
+    
+    return true;
+}
+
+/* This function is called once from main(), it handles
+   the decoding and checks the fields. */
+bool check_alltypes(pb_istream_t *stream, int mode)
+{
+    /* Values for use from callbacks through pointers. */
+    uint32_t    req_fixed32     = 1008;
+    int32_t     req_sfixed32    = -1009;
+    float       req_float       = 1010.0f;
+    uint64_t    req_fixed64     = 1011;
+    int64_t     req_sfixed64    = -1012;
+    double      req_double      = 1013.0;
+    SubMessage  req_submsg      = {"1016", 1016};
+    
+    int32_t     rep_int32[5]    = {0, 0, 0, 0, -2001};
+    int32_t     rep_int64[5]    = {0, 0, 0, 0, -2002};
+    int32_t     rep_uint32[5]   = {0, 0, 0, 0,  2003};
+    int32_t     rep_uint64[5]   = {0, 0, 0, 0,  2004};
+    int32_t     rep_sint32[5]   = {0, 0, 0, 0, -2005};
+    int32_t     rep_sint64[5]   = {0, 0, 0, 0, -2006};
+    int32_t     rep_bool[5]     = {false, false, false, false, true};
+    uint32_t    rep_fixed32[5]  = {0, 0, 0, 0,  2008};
+    int32_t     rep_sfixed32[5] = {0, 0, 0, 0, -2009};
+    float       rep_float[5]    = {0, 0, 0, 0,  2010.0f};
+    uint64_t    rep_fixed64[5]  = {0, 0, 0, 0,  2011};
+    int64_t     rep_sfixed64[5] = {0, 0, 0, 0, -2012};
+    double      rep_double[5]   = {0, 0, 0, 0,  2013.0};
+    char*       rep_string[5]   = {"", "", "", "", "2014"};
+    char*       rep_bytes[5]    = {"", "", "", "", "2015"};
+    SubMessage  rep_submsg[5]   = {{"", 0, 0, 3},
+                                   {"", 0, 0, 3},
+                                   {"", 0, 0, 3},
+                                   {"", 0, 0, 3},
+                                   {"2016", 2016, true, 2016}};
+    int32_t     rep_enum[5]     = {0, 0, 0, 0, MyEnum_Truth};
+    
+    uint32_t    opt_fixed32     = 3048;
+    int32_t     opt_sfixed32    = 3049;
+    float       opt_float       = 3050.0f;
+    uint64_t    opt_fixed64     = 3051;
+    int64_t     opt_sfixed64    = 3052;
+    double      opt_double      = 3053.0f;
+    SubMessage  opt_submsg      = {"3056", 3056};
+    
+    /* Bind callbacks for required fields */
+    AllTypes alltypes;
+    
+    /* Fill with garbage to better detect initialization errors */
+    memset(&alltypes, 0xAA, sizeof(alltypes));
+    alltypes.extensions = 0;
+    
+    alltypes.req_int32.funcs.decode = &read_varint;
+    alltypes.req_int32.arg = (void*)-1001;
+    
+    alltypes.req_int64.funcs.decode = &read_varint;
+    alltypes.req_int64.arg = (void*)-1002;
+    
+    alltypes.req_uint32.funcs.decode = &read_varint;
+    alltypes.req_uint32.arg = (void*)1003;
+
+    alltypes.req_uint32.funcs.decode = &read_varint;
+    alltypes.req_uint32.arg = (void*)1003;    
+    
+    alltypes.req_uint64.funcs.decode = &read_varint;
+    alltypes.req_uint64.arg = (void*)1004;
+    
+    alltypes.req_sint32.funcs.decode = &read_svarint;
+    alltypes.req_sint32.arg = (void*)-1005;   
+    
+    alltypes.req_sint64.funcs.decode = &read_svarint;
+    alltypes.req_sint64.arg = (void*)-1006;   
+    
+    alltypes.req_bool.funcs.decode = &read_varint;
+    alltypes.req_bool.arg = (void*)true;   
+    
+    alltypes.req_fixed32.funcs.decode = &read_fixed32;
+    alltypes.req_fixed32.arg = &req_fixed32;
+    
+    alltypes.req_sfixed32.funcs.decode = &read_fixed32;
+    alltypes.req_sfixed32.arg = &req_sfixed32;
+    
+    alltypes.req_float.funcs.decode = &read_fixed32;
+    alltypes.req_float.arg = &req_float;
+    
+    alltypes.req_fixed64.funcs.decode = &read_fixed64;
+    alltypes.req_fixed64.arg = &req_fixed64;
+    
+    alltypes.req_sfixed64.funcs.decode = &read_fixed64;
+    alltypes.req_sfixed64.arg = &req_sfixed64;
+    
+    alltypes.req_double.funcs.decode = &read_fixed64;
+    alltypes.req_double.arg = &req_double;
+    
+    alltypes.req_string.funcs.decode = &read_string;
+    alltypes.req_string.arg = "1014";
+    
+    alltypes.req_bytes.funcs.decode = &read_string;
+    alltypes.req_bytes.arg = "1015";
+    
+    alltypes.req_submsg.funcs.decode = &read_submsg;
+    alltypes.req_submsg.arg = &req_submsg;
+    
+    alltypes.req_enum.funcs.decode = &read_varint;
+    alltypes.req_enum.arg = (void*)MyEnum_Truth;
+    
+    alltypes.req_emptymsg.funcs.decode = &read_emptymsg;
+    
+    /* Bind callbacks for repeated fields */
+    alltypes.rep_int32.funcs.decode = &read_repeated_varint;
+    alltypes.rep_int32.arg = rep_int32;
+    
+    alltypes.rep_int64.funcs.decode = &read_repeated_varint;
+    alltypes.rep_int64.arg = rep_int64;
+    
+    alltypes.rep_uint32.funcs.decode = &read_repeated_varint;
+    alltypes.rep_uint32.arg = rep_uint32;
+    
+    alltypes.rep_uint64.funcs.decode = &read_repeated_varint;
+    alltypes.rep_uint64.arg = rep_uint64;
+    
+    alltypes.rep_sint32.funcs.decode = &read_repeated_svarint;
+    alltypes.rep_sint32.arg = rep_sint32;
+    
+    alltypes.rep_sint64.funcs.decode = &read_repeated_svarint;
+    alltypes.rep_sint64.arg = rep_sint64;
+    
+    alltypes.rep_bool.funcs.decode = &read_repeated_varint;
+    alltypes.rep_bool.arg = rep_bool;
+    
+    alltypes.rep_fixed32.funcs.decode = &read_repeated_fixed32;
+    alltypes.rep_fixed32.arg = rep_fixed32;
+    
+    alltypes.rep_sfixed32.funcs.decode = &read_repeated_fixed32;
+    alltypes.rep_sfixed32.arg = rep_sfixed32;
+    
+    alltypes.rep_float.funcs.decode = &read_repeated_fixed32;
+    alltypes.rep_float.arg = rep_float;
+    
+    alltypes.rep_fixed64.funcs.decode = &read_repeated_fixed64;
+    alltypes.rep_fixed64.arg = rep_fixed64;
+    
+    alltypes.rep_sfixed64.funcs.decode = &read_repeated_fixed64;
+    alltypes.rep_sfixed64.arg = rep_sfixed64;
+    
+    alltypes.rep_double.funcs.decode = &read_repeated_fixed64;
+    alltypes.rep_double.arg = rep_double;
+    
+    alltypes.rep_string.funcs.decode = &read_repeated_string;
+    alltypes.rep_string.arg = rep_string;
+    
+    alltypes.rep_bytes.funcs.decode = &read_repeated_string;
+    alltypes.rep_bytes.arg = rep_bytes;
+    
+    alltypes.rep_submsg.funcs.decode = &read_repeated_submsg;
+    alltypes.rep_submsg.arg = rep_submsg;
+    
+    alltypes.rep_enum.funcs.decode = &read_repeated_varint;
+    alltypes.rep_enum.arg = rep_enum;
+    
+    alltypes.rep_emptymsg.funcs.decode = &read_emptymsg;
+    
+    alltypes.req_limits.funcs.decode = &read_limits;
+    
+    alltypes.end.funcs.decode = &read_varint;
+    alltypes.end.arg = (void*)1099;
+    
+    /* Bind callbacks for optional fields */
+    if (mode == 1)
+    {
+        alltypes.opt_int32.funcs.decode = &read_varint;
+        alltypes.opt_int32.arg = (void*)3041;
+        
+        alltypes.opt_int64.funcs.decode = &read_varint;
+        alltypes.opt_int64.arg = (void*)3042;
+        
+        alltypes.opt_uint32.funcs.decode = &read_varint;
+        alltypes.opt_uint32.arg = (void*)3043;
+        
+        alltypes.opt_uint64.funcs.decode = &read_varint;
+        alltypes.opt_uint64.arg = (void*)3044;
+        
+        alltypes.opt_sint32.funcs.decode = &read_svarint;
+        alltypes.opt_sint32.arg = (void*)3045;
+        
+        alltypes.opt_sint64.funcs.decode = &read_svarint;
+        alltypes.opt_sint64.arg = (void*)3046;
+        
+        alltypes.opt_bool.funcs.decode = &read_varint;
+        alltypes.opt_bool.arg = (void*)true;
+
+        alltypes.opt_fixed32.funcs.decode = &read_fixed32;
+        alltypes.opt_fixed32.arg = &opt_fixed32;
+        
+        alltypes.opt_sfixed32.funcs.decode = &read_fixed32;
+        alltypes.opt_sfixed32.arg = &opt_sfixed32;
+        
+        alltypes.opt_float.funcs.decode = &read_fixed32;
+        alltypes.opt_float.arg = &opt_float;
+        
+        alltypes.opt_fixed64.funcs.decode = &read_fixed64;
+        alltypes.opt_fixed64.arg = &opt_fixed64;
+        
+        alltypes.opt_sfixed64.funcs.decode = &read_fixed64;
+        alltypes.opt_sfixed64.arg = &opt_sfixed64;
+        
+        alltypes.opt_double.funcs.decode = &read_fixed64;
+        alltypes.opt_double.arg = &opt_double;
+        
+        alltypes.opt_string.funcs.decode = &read_string;
+        alltypes.opt_string.arg = "3054";
+        
+        alltypes.opt_bytes.funcs.decode = &read_string;
+        alltypes.opt_bytes.arg = "3055";
+        
+        alltypes.opt_submsg.funcs.decode = &read_submsg;
+        alltypes.opt_submsg.arg = &opt_submsg;
+        
+        alltypes.opt_enum.funcs.decode = &read_varint;
+        alltypes.opt_enum.arg = (void*)MyEnum_Truth;
+        
+        alltypes.opt_emptymsg.funcs.decode = &read_emptymsg;
+    }
+    
+    return pb_decode(stream, AllTypes_fields, &alltypes);
+}
+
+int main(int argc, char **argv)
+{
+    uint8_t buffer[1024];
+    size_t count;
+    pb_istream_t stream;
+
+    /* Whether to expect the optional values or the default values. */
+    int mode = (argc > 1) ? atoi(argv[1]) : 0;
+    
+    /* Read the data into buffer */
+    SET_BINARY_MODE(stdin);
+    count = fread(buffer, 1, sizeof(buffer), stdin);
+    
+    /* Construct a pb_istream_t for reading from the buffer */
+    stream = pb_istream_from_buffer(buffer, count);
+    
+    /* Decode and print out the stuff */
+    if (!check_alltypes(&stream, mode))
+    {
+        printf("Parsing failed: %s\n", PB_GET_ERROR(&stream));
+        return 1;
+    } else {
+        return 0;
+    }
+}
--- a/tests/alltypes_callback/encode_alltypes_callback.c
+++ b/tests/alltypes_callback/encode_alltypes_callback.c
@@ -0,0 +1,397 @@
+/* Attempts to test all the datatypes supported by ProtoBuf when used as callback fields.
+ * Note that normally there would be no reason to use callback fields for this,
+ * because each encoder defined here only gives a single field.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <pb_encode.h>
+#include "alltypes.pb.h"
+#include "test_helpers.h"
+
+static bool write_varint(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)
+{
+    return pb_encode_tag_for_field(stream, field) &&
+           pb_encode_varint(stream, (long)*arg);
+}
+
+static bool write_svarint(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)
+{
+    return pb_encode_tag_for_field(stream, field) &&
+           pb_encode_svarint(stream, (long)*arg);
+}
+
+static bool write_fixed32(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)
+{
+    return pb_encode_tag_for_field(stream, field) &&
+           pb_encode_fixed32(stream, *arg);
+}
+
+static bool write_fixed64(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)
+{
+    return pb_encode_tag_for_field(stream, field) &&
+           pb_encode_fixed64(stream, *arg);
+}
+
+static bool write_string(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)
+{
+    return pb_encode_tag_for_field(stream, field) &&
+           pb_encode_string(stream, *arg, strlen(*arg));
+}
+
+static bool write_submsg(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)
+{
+   
+    return pb_encode_tag_for_field(stream, field) &&
+           pb_encode_submessage(stream, SubMessage_fields, *arg);
+}
+
+static bool write_emptymsg(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)
+{
+    EmptyMessage emptymsg = {0};
+    return pb_encode_tag_for_field(stream, field) &&
+           pb_encode_submessage(stream, EmptyMessage_fields, &emptymsg);
+}
+
+static bool write_repeated_varint(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)
+{
+    return pb_encode_tag_for_field(stream, field) &&
+           pb_encode_varint(stream, 0) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_varint(stream, 0) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_varint(stream, 0) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_varint(stream, 0) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_varint(stream, (long)*arg);
+}
+
+static bool write_repeated_svarint(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)
+{
+    return pb_encode_tag_for_field(stream, field) &&
+           pb_encode_svarint(stream, 0) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_svarint(stream, 0) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_svarint(stream, 0) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_svarint(stream, 0) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_svarint(stream, (long)*arg);
+}
+
+static bool write_repeated_fixed32(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)
+{
+    uint32_t dummy = 0;
+
+    /* Make it a packed field */
+    return pb_encode_tag(stream, PB_WT_STRING, field->tag) &&
+           pb_encode_varint(stream, 5 * 4) && /* Number of bytes */
+           pb_encode_fixed32(stream, &dummy) &&
+           pb_encode_fixed32(stream, &dummy) &&
+           pb_encode_fixed32(stream, &dummy) &&
+           pb_encode_fixed32(stream, &dummy) &&
+           pb_encode_fixed32(stream, *arg);
+}
+
+static bool write_repeated_fixed64(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)
+{
+    uint64_t dummy = 0;
+
+    /* Make it a packed field */
+    return pb_encode_tag(stream, PB_WT_STRING, field->tag) &&
+           pb_encode_varint(stream, 5 * 8) && /* Number of bytes */
+           pb_encode_fixed64(stream, &dummy) &&
+           pb_encode_fixed64(stream, &dummy) &&
+           pb_encode_fixed64(stream, &dummy) &&
+           pb_encode_fixed64(stream, &dummy) &&
+           pb_encode_fixed64(stream, *arg);
+}
+
+static bool write_repeated_string(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)
+{
+    return pb_encode_tag_for_field(stream, field) &&
+           pb_encode_string(stream, 0, 0) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_string(stream, 0, 0) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_string(stream, 0, 0) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_string(stream, 0, 0) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_string(stream, *arg, strlen(*arg));
+}
+
+static bool write_repeated_submsg(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)
+{
+    SubMessage dummy = {""};
+
+    return pb_encode_tag_for_field(stream, field) &&
+           pb_encode_submessage(stream, SubMessage_fields, &dummy) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_submessage(stream, SubMessage_fields, &dummy) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_submessage(stream, SubMessage_fields, &dummy) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_submessage(stream, SubMessage_fields, &dummy) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_submessage(stream, SubMessage_fields, *arg);
+}
+
+static bool write_limits(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)
+{
+    Limits limits = {0};
+    limits.int32_min  = INT32_MIN;
+    limits.int32_max  = INT32_MAX;
+    limits.uint32_min = 0;
+    limits.uint32_max = UINT32_MAX;
+    limits.int64_min  = INT64_MIN;
+    limits.int64_max  = INT64_MAX;
+    limits.uint64_min = 0;
+    limits.uint64_max = UINT64_MAX;
+    limits.enum_min   = HugeEnum_Negative;
+    limits.enum_max   = HugeEnum_Positive;
+   
+    return pb_encode_tag_for_field(stream, field) &&
+           pb_encode_submessage(stream, Limits_fields, &limits);
+}
+
+static bool write_repeated_emptymsg(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)
+{
+    EmptyMessage emptymsg = {0};
+    return pb_encode_tag_for_field(stream, field) &&
+           pb_encode_submessage(stream, EmptyMessage_fields, &emptymsg) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_submessage(stream, EmptyMessage_fields, &emptymsg) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_submessage(stream, EmptyMessage_fields, &emptymsg) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_submessage(stream, EmptyMessage_fields, &emptymsg) &&
+           pb_encode_tag_for_field(stream, field) &&
+           pb_encode_submessage(stream, EmptyMessage_fields, &emptymsg);
+}
+
+int main(int argc, char **argv)
+{
+    int mode = (argc > 1) ? atoi(argv[1]) : 0;
+
+    /* Values for use from callbacks through pointers. */
+    uint32_t    req_fixed32     = 1008;
+    int32_t     req_sfixed32    = -1009;
+    float       req_float       = 1010.0f;
+    uint64_t    req_fixed64     = 1011;
+    int64_t     req_sfixed64    = -1012;
+    double      req_double      = 1013.0;
+    SubMessage  req_submsg      = {"1016", 1016};
+    
+    uint32_t    rep_fixed32     = 2008;
+    int32_t     rep_sfixed32    = -2009;
+    float       rep_float       = 2010.0f;
+    uint64_t    rep_fixed64     = 2011;
+    int64_t     rep_sfixed64    = -2012;
+    double      rep_double      = 2013.0;
+    SubMessage  rep_submsg      = {"2016", 2016, true, 2016};
+    
+    uint32_t    opt_fixed32     = 3048;
+    int32_t     opt_sfixed32    = 3049;
+    float       opt_float       = 3050.0f;
+    uint64_t    opt_fixed64     = 3051;
+    int64_t     opt_sfixed64    = 3052;
+    double      opt_double      = 3053.0f;
+    SubMessage  opt_submsg      = {"3056", 3056};
+    
+    /* Bind callbacks for required fields */
+    AllTypes alltypes = {{{0}}};
+    
+    alltypes.req_int32.funcs.encode = &write_varint;
+    alltypes.req_int32.arg = (void*)-1001;
+    
+    alltypes.req_int64.funcs.encode = &write_varint;
+    alltypes.req_int64.arg = (void*)-1002;
+    
+    alltypes.req_uint32.funcs.encode = &write_varint;
+    alltypes.req_uint32.arg = (void*)1003;
+
+    alltypes.req_uint32.funcs.encode = &write_varint;
+    alltypes.req_uint32.arg = (void*)1003;    
+    
+    alltypes.req_uint64.funcs.encode = &write_varint;
+    alltypes.req_uint64.arg = (void*)1004;
+    
+    alltypes.req_sint32.funcs.encode = &write_svarint;
+    alltypes.req_sint32.arg = (void*)-1005;   
+    
+    alltypes.req_sint64.funcs.encode = &write_svarint;
+    alltypes.req_sint64.arg = (void*)-1006;   
+    
+    alltypes.req_bool.funcs.encode = &write_varint;
+    alltypes.req_bool.arg = (void*)true;   
+    
+    alltypes.req_fixed32.funcs.encode = &write_fixed32;
+    alltypes.req_fixed32.arg = &req_fixed32;
+    
+    alltypes.req_sfixed32.funcs.encode = &write_fixed32;
+    alltypes.req_sfixed32.arg = &req_sfixed32;
+    
+    alltypes.req_float.funcs.encode = &write_fixed32;
+    alltypes.req_float.arg = &req_float;
+    
+    alltypes.req_fixed64.funcs.encode = &write_fixed64;
+    alltypes.req_fixed64.arg = &req_fixed64;
+    
+    alltypes.req_sfixed64.funcs.encode = &write_fixed64;
+    alltypes.req_sfixed64.arg = &req_sfixed64;
+    
+    alltypes.req_double.funcs.encode = &write_fixed64;
+    alltypes.req_double.arg = &req_double;
+    
+    alltypes.req_string.funcs.encode = &write_string;
+    alltypes.req_string.arg = "1014";
+    
+    alltypes.req_bytes.funcs.encode = &write_string;
+    alltypes.req_bytes.arg = "1015";
+    
+    alltypes.req_submsg.funcs.encode = &write_submsg;
+    alltypes.req_submsg.arg = &req_submsg;
+    
+    alltypes.req_enum.funcs.encode = &write_varint;
+    alltypes.req_enum.arg = (void*)MyEnum_Truth;
+    
+    alltypes.req_emptymsg.funcs.encode = &write_emptymsg;
+    
+    /* Bind callbacks for repeated fields */
+    alltypes.rep_int32.funcs.encode = &write_repeated_varint;
+    alltypes.rep_int32.arg = (void*)-2001;
+    
+    alltypes.rep_int64.funcs.encode = &write_repeated_varint;
+    alltypes.rep_int64.arg = (void*)-2002;
+    
+    alltypes.rep_uint32.funcs.encode = &write_repeated_varint;
+    alltypes.rep_uint32.arg = (void*)2003;
+    
+    alltypes.rep_uint64.funcs.encode = &write_repeated_varint;
+    alltypes.rep_uint64.arg = (void*)2004;
+    
+    alltypes.rep_sint32.funcs.encode = &write_repeated_svarint;
+    alltypes.rep_sint32.arg = (void*)-2005;
+    
+    alltypes.rep_sint64.funcs.encode = &write_repeated_svarint;
+    alltypes.rep_sint64.arg = (void*)-2006;
+    
+    alltypes.rep_bool.funcs.encode = &write_repeated_varint;
+    alltypes.rep_bool.arg = (void*)true;
+    
+    alltypes.rep_fixed32.funcs.encode = &write_repeated_fixed32;
+    alltypes.rep_fixed32.arg = &rep_fixed32;
+
+    alltypes.rep_sfixed32.funcs.encode = &write_repeated_fixed32;
+    alltypes.rep_sfixed32.arg = &rep_sfixed32;
+    
+    alltypes.rep_float.funcs.encode = &write_repeated_fixed32;
+    alltypes.rep_float.arg = &rep_float;
+    
+    alltypes.rep_fixed64.funcs.encode = &write_repeated_fixed64;
+    alltypes.rep_fixed64.arg = &rep_fixed64;
+
+    alltypes.rep_sfixed64.funcs.encode = &write_repeated_fixed64;
+    alltypes.rep_sfixed64.arg = &rep_sfixed64;
+    
+    alltypes.rep_double.funcs.encode = &write_repeated_fixed64;
+    alltypes.rep_double.arg = &rep_double;
+    
+    alltypes.rep_string.funcs.encode = &write_repeated_string;
+    alltypes.rep_string.arg = "2014";
+    
+    alltypes.rep_bytes.funcs.encode = &write_repeated_string;
+    alltypes.rep_bytes.arg = "2015";
+    
+    alltypes.rep_submsg.funcs.encode = &write_repeated_submsg;
+    alltypes.rep_submsg.arg = &rep_submsg;
+    
+    alltypes.rep_enum.funcs.encode = &write_repeated_varint;
+    alltypes.rep_enum.arg = (void*)MyEnum_Truth;
+    
+    alltypes.rep_emptymsg.funcs.encode = &write_repeated_emptymsg;
+    
+    alltypes.req_limits.funcs.encode = &write_limits;
+    
+    /* Bind callbacks for optional fields */
+    if (mode != 0)
+    {
+        alltypes.opt_int32.funcs.encode = &write_varint;
+        alltypes.opt_int32.arg = (void*)3041;
+        
+        alltypes.opt_int64.funcs.encode = &write_varint;
+        alltypes.opt_int64.arg = (void*)3042;
+        
+        alltypes.opt_uint32.funcs.encode = &write_varint;
+        alltypes.opt_uint32.arg = (void*)3043;
+        
+        alltypes.opt_uint64.funcs.encode = &write_varint;
+        alltypes.opt_uint64.arg = (void*)3044;
+        
+        alltypes.opt_sint32.funcs.encode = &write_svarint;
+        alltypes.opt_sint32.arg = (void*)3045;
+        
+        alltypes.opt_sint64.funcs.encode = &write_svarint;
+        alltypes.opt_sint64.arg = (void*)3046;
+        
+        alltypes.opt_bool.funcs.encode = &write_varint;
+        alltypes.opt_bool.arg = (void*)true;
+
+        alltypes.opt_fixed32.funcs.encode = &write_fixed32;
+        alltypes.opt_fixed32.arg = &opt_fixed32;
+        
+        alltypes.opt_sfixed32.funcs.encode = &write_fixed32;
+        alltypes.opt_sfixed32.arg = &opt_sfixed32;
+        
+        alltypes.opt_float.funcs.encode = &write_fixed32;
+        alltypes.opt_float.arg = &opt_float;
+        
+        alltypes.opt_fixed64.funcs.encode = &write_fixed64;
+        alltypes.opt_fixed64.arg = &opt_fixed64;
+        
+        alltypes.opt_sfixed64.funcs.encode = &write_fixed64;
+        alltypes.opt_sfixed64.arg = &opt_sfixed64;
+        
+        alltypes.opt_double.funcs.encode = &write_fixed64;
+        alltypes.opt_double.arg = &opt_double;
+        
+        alltypes.opt_string.funcs.encode = &write_string;
+        alltypes.opt_string.arg = "3054";
+        
+        alltypes.opt_bytes.funcs.encode = &write_string;
+        alltypes.opt_bytes.arg = "3055";
+        
+        alltypes.opt_submsg.funcs.encode = &write_submsg;
+        alltypes.opt_submsg.arg = &opt_submsg;
+        
+        alltypes.opt_enum.funcs.encode = &write_varint;
+        alltypes.opt_enum.arg = (void*)MyEnum_Truth;
+        
+        alltypes.opt_emptymsg.funcs.encode = &write_emptymsg;
+    }
+    
+    alltypes.end.funcs.encode = &write_varint;
+    alltypes.end.arg = (void*)1099;
+    
+    {
+        uint8_t buffer[2048];
+        pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
+        
+        /* Now encode it and check if we succeeded. */
+        if (pb_encode(&stream, AllTypes_fields, &alltypes))
+        {
+            SET_BINARY_MODE(stdout);
+            fwrite(buffer, 1, stream.bytes_written, stdout);
+            return 0; /* Success */
+        }
+        else
+        {
+            fprintf(stderr, "Encoding failed: %s\n", PB_GET_ERROR(&stream));
+            return 1; /* Failure */
+        }
+    }
+}
--- a/tests/alltypes_pointer/SConscript
+++ b/tests/alltypes_pointer/SConscript
@@ -3,21 +3,46 @@

 Import("env")

+# We need our own pb_decode.o for the malloc support
+env = env.Clone()
+env.Append(CPPDEFINES = {'PB_ENABLE_MALLOC': 1});
+
+# Disable libmudflap, because it will confuse valgrind
+# and other memory leak detection tools.
+if '-fmudflap' in env["CCFLAGS"]:
+    env["CCFLAGS"].remove("-fmudflap")
+    env["LINKFLAGS"].remove("-fmudflap")
+    env["LIBS"].remove("mudflap")
+
+strict = env.Clone()
+strict.Append(CFLAGS = strict['CORECFLAGS'])
+strict.Object("pb_decode_with_malloc.o", "$NANOPB/pb_decode.c")
+strict.Object("pb_encode_with_malloc.o", "$NANOPB/pb_encode.c")
+
 c = Copy("$TARGET", "$SOURCE")
 env.Command("alltypes.proto", "#alltypes/alltypes.proto", c)

 env.NanopbProto(["alltypes", "alltypes.options"])
-enc = env.Program(["encode_alltypes_pointer.c", "alltypes.pb.c", "$COMMON/pb_encode.o"])
+enc = env.Program(["encode_alltypes_pointer.c", "alltypes.pb.c", "pb_encode_with_malloc.o"])
+dec = env.Program(["decode_alltypes_pointer.c", "alltypes.pb.c", "pb_decode_with_malloc.o"])

-refdec = "$BUILD/alltypes/decode_alltypes$PROGSUFFIX"
-
-# Encode and compare results
+# Encode and compare results to non-pointer alltypes test case
 env.RunTest(enc)
-env.RunTest("decode_alltypes.output", [refdec, "encode_alltypes_pointer.output"])
 env.Compare(["encode_alltypes_pointer.output", "$BUILD/alltypes/encode_alltypes.output"])

+# Decode (under valgrind if available)
+valgrind = env.WhereIs('valgrind')
+kwargs = {}
+if valgrind:
+    kwargs['COMMAND'] = valgrind
+    kwargs['ARGS'] = ["-q", dec[0].abspath]
+
+env.RunTest("decode_alltypes.output", [dec, "encode_alltypes_pointer.output"], **kwargs)
+
 # Do the same thing with the optional fields present
 env.RunTest("optionals.output", enc, ARGS = ['1'])
-env.RunTest("optionals.decout", [refdec, "optionals.output"], ARGS = ['1'])
 env.Compare(["optionals.output", "$BUILD/alltypes/optionals.output"])

+kwargs['ARGS'] = kwargs.get('ARGS', []) + ['1']
+env.RunTest("optionals.decout", [dec, "optionals.output"], **kwargs)
+
--- a/tests/alltypes_pointer/decode_alltypes_pointer.c
+++ b/tests/alltypes_pointer/decode_alltypes_pointer.c
@@ -0,0 +1,174 @@
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <pb_decode.h>
+#include "alltypes.pb.h"
+#include "test_helpers.h"
+
+#define TEST(x) if (!(x)) { \
+    fprintf(stderr, "Test " #x " failed.\n"); \
+    status = false; \
+    }
+
+/* This function is called once from main(), it handles
+   the decoding and checks the fields. */
+bool check_alltypes(pb_istream_t *stream, int mode)
+{
+    bool status = true;
+    AllTypes alltypes;
+    
+    /* Fill with garbage to better detect initialization errors */
+    memset(&alltypes, 0xAA, sizeof(alltypes));
+    alltypes.extensions = 0;
+    
+    if (!pb_decode(stream, AllTypes_fields, &alltypes))
+        return false;
+    
+    TEST(alltypes.req_int32     && *alltypes.req_int32         == -1001);
+    TEST(alltypes.req_int64     && *alltypes.req_int64         == -1002);
+    TEST(alltypes.req_uint32    && *alltypes.req_uint32        == 1003);
+    TEST(alltypes.req_uint64    && *alltypes.req_uint64        == 1004);
+    TEST(alltypes.req_sint32    && *alltypes.req_sint32        == -1005);
+    TEST(alltypes.req_sint64    && *alltypes.req_sint64        == -1006);
+    TEST(alltypes.req_bool      && *alltypes.req_bool          == true);
+    
+    TEST(alltypes.req_fixed32   && *alltypes.req_fixed32       == 1008);
+    TEST(alltypes.req_sfixed32  && *alltypes.req_sfixed32      == -1009);
+    TEST(alltypes.req_float     && *alltypes.req_float         == 1010.0f);
+    
+    TEST(alltypes.req_fixed64   && *alltypes.req_fixed64       == 1011);
+    TEST(alltypes.req_sfixed64  && *alltypes.req_sfixed64      == -1012);
+    TEST(alltypes.req_double    && *alltypes.req_double        == 1013.0f);
+    
+    TEST(alltypes.req_string    && strcmp(alltypes.req_string, "1014") == 0);
+    TEST(alltypes.req_bytes     && alltypes.req_bytes->size == 4);
+    TEST(alltypes.req_bytes     && memcmp(&alltypes.req_bytes->bytes, "1015", 4) == 0);
+    TEST(alltypes.req_submsg    && alltypes.req_submsg->substuff1
+                                && strcmp(alltypes.req_submsg->substuff1, "1016") == 0);
+    TEST(alltypes.req_submsg    && alltypes.req_submsg->substuff2
+                                && *alltypes.req_submsg->substuff2 == 1016);
+    TEST(*alltypes.req_enum == MyEnum_Truth);
+
+    TEST(alltypes.rep_int32_count == 5 && alltypes.rep_int32[4] == -2001 && alltypes.rep_int32[0] == 0);
+    TEST(alltypes.rep_int64_count == 5 && alltypes.rep_int64[4] == -2002 && alltypes.rep_int64[0] == 0);
+    TEST(alltypes.rep_uint32_count == 5 && alltypes.rep_uint32[4] == 2003 && alltypes.rep_uint32[0] == 0);
+    TEST(alltypes.rep_uint64_count == 5 && alltypes.rep_uint64[4] == 2004 && alltypes.rep_uint64[0] == 0);
+    TEST(alltypes.rep_sint32_count == 5 && alltypes.rep_sint32[4] == -2005 && alltypes.rep_sint32[0] == 0);
+    TEST(alltypes.rep_sint64_count == 5 && alltypes.rep_sint64[4] == -2006 && alltypes.rep_sint64[0] == 0);
+    TEST(alltypes.rep_bool_count == 5 && alltypes.rep_bool[4] == true && alltypes.rep_bool[0] == false);
+    
+    TEST(alltypes.rep_fixed32_count == 5 && alltypes.rep_fixed32[4] == 2008 && alltypes.rep_fixed32[0] == 0);
+    TEST(alltypes.rep_sfixed32_count == 5 && alltypes.rep_sfixed32[4] == -2009 && alltypes.rep_sfixed32[0] == 0);
+    TEST(alltypes.rep_float_count == 5 && alltypes.rep_float[4] == 2010.0f && alltypes.rep_float[0] == 0.0f);
+    
+    TEST(alltypes.rep_fixed64_count == 5 && alltypes.rep_fixed64[4] == 2011 && alltypes.rep_fixed64[0] == 0);
+    TEST(alltypes.rep_sfixed64_count == 5 && alltypes.rep_sfixed64[4] == -2012 && alltypes.rep_sfixed64[0] == 0);
+    TEST(alltypes.rep_double_count == 5 && alltypes.rep_double[4] == 2013.0 && alltypes.rep_double[0] == 0.0);
+    
+    TEST(alltypes.rep_string_count == 5 && strcmp(alltypes.rep_string[4], "2014") == 0 && alltypes.rep_string[0][0] == '\0');
+    TEST(alltypes.rep_bytes_count == 5 && alltypes.rep_bytes[4]->size == 4 && alltypes.rep_bytes[0]->size == 0);
+    TEST(memcmp(&alltypes.rep_bytes[4]->bytes, "2015", 4) == 0);
+
+    TEST(alltypes.rep_submsg_count == 5);
+    TEST(strcmp(alltypes.rep_submsg[4].substuff1, "2016") == 0 && alltypes.rep_submsg[0].substuff1[0] == '\0');
+    TEST(*alltypes.rep_submsg[4].substuff2 == 2016 && *alltypes.rep_submsg[0].substuff2 == 0);
+    TEST(*alltypes.rep_submsg[4].substuff3 == 2016 && alltypes.rep_submsg[0].substuff3 == NULL);
+    
+    TEST(alltypes.rep_enum_count == 5 && alltypes.rep_enum[4] == MyEnum_Truth && alltypes.rep_enum[0] == MyEnum_Zero);
+    TEST(alltypes.rep_emptymsg_count == 5);
+
+    if (mode == 0)
+    {
+        /* Expect that optional values are not present */
+        TEST(alltypes.opt_int32         == NULL);
+        TEST(alltypes.opt_int64         == NULL);
+        TEST(alltypes.opt_uint32        == NULL);
+        TEST(alltypes.opt_uint64        == NULL);
+        TEST(alltypes.opt_sint32        == NULL);
+        TEST(alltypes.opt_sint64        == NULL);
+        TEST(alltypes.opt_bool          == NULL);
+        
+        TEST(alltypes.opt_fixed32       == NULL);
+        TEST(alltypes.opt_sfixed32      == NULL);
+        TEST(alltypes.opt_float         == NULL);
+        TEST(alltypes.opt_fixed64       == NULL);
+        TEST(alltypes.opt_sfixed64      == NULL);
+        TEST(alltypes.opt_double        == NULL);
+        
+        TEST(alltypes.opt_string        == NULL);
+        TEST(alltypes.opt_bytes         == NULL);
+        TEST(alltypes.opt_submsg        == NULL);
+        TEST(alltypes.opt_enum          == NULL);
+    }
+    else
+    {
+        /* Expect filled-in values */
+        TEST(alltypes.opt_int32 && *alltypes.opt_int32      == 3041);
+        TEST(alltypes.opt_int64 && *alltypes.opt_int64      == 3042);
+        TEST(alltypes.opt_uint32 && *alltypes.opt_uint32    == 3043);
+        TEST(alltypes.opt_uint64 && *alltypes.opt_uint64    == 3044);
+        TEST(alltypes.opt_sint32 && *alltypes.opt_sint32    == 3045);
+        TEST(alltypes.opt_sint64 && *alltypes.opt_sint64    == 3046);
+        TEST(alltypes.opt_bool && *alltypes.opt_bool        == true);
+        
+        TEST(alltypes.opt_fixed32 && *alltypes.opt_fixed32  == 3048);
+        TEST(alltypes.opt_sfixed32 && *alltypes.opt_sfixed32== 3049);
+        TEST(alltypes.opt_float && *alltypes.opt_float      == 3050.0f);
+        TEST(alltypes.opt_fixed64 && *alltypes.opt_fixed64  == 3051);
+        TEST(alltypes.opt_sfixed64 && *alltypes.opt_sfixed64== 3052);
+        TEST(alltypes.opt_double && *alltypes.opt_double    == 3053.0);
+        
+        TEST(alltypes.opt_string && strcmp(alltypes.opt_string, "3054") == 0);
+        TEST(alltypes.opt_bytes && alltypes.opt_bytes->size == 4);
+        TEST(alltypes.opt_bytes && memcmp(&alltypes.opt_bytes->bytes, "3055", 4) == 0);
+        TEST(alltypes.opt_submsg && strcmp(alltypes.opt_submsg->substuff1, "3056") == 0);
+        TEST(alltypes.opt_submsg && *alltypes.opt_submsg->substuff2 == 3056);
+        TEST(alltypes.opt_enum && *alltypes.opt_enum == MyEnum_Truth);
+        TEST(alltypes.opt_emptymsg);
+    }
+    
+    TEST(alltypes.req_limits->int32_min && *alltypes.req_limits->int32_min   == INT32_MIN);
+    TEST(alltypes.req_limits->int32_max && *alltypes.req_limits->int32_max   == INT32_MAX);
+    TEST(alltypes.req_limits->uint32_min && *alltypes.req_limits->uint32_min == 0);
+    TEST(alltypes.req_limits->uint32_max && *alltypes.req_limits->uint32_max == UINT32_MAX);
+    TEST(alltypes.req_limits->int64_min && *alltypes.req_limits->int64_min   == INT64_MIN);
+    TEST(alltypes.req_limits->int64_max && *alltypes.req_limits->int64_max   == INT64_MAX);
+    TEST(alltypes.req_limits->uint64_min && *alltypes.req_limits->uint64_min == 0);
+    TEST(alltypes.req_limits->uint64_max && *alltypes.req_limits->uint64_max == UINT64_MAX);
+    TEST(alltypes.req_limits->enum_min && *alltypes.req_limits->enum_min     == HugeEnum_Negative);
+    TEST(alltypes.req_limits->enum_max && *alltypes.req_limits->enum_max     == HugeEnum_Positive);
+    
+    TEST(alltypes.end && *alltypes.end == 1099);
+
+    pb_release(AllTypes_fields, &alltypes);
+
+    return status;
+}
+
+int main(int argc, char **argv)
+{
+    uint8_t buffer[1024];
+    size_t count;
+    pb_istream_t stream;
+    
+    /* Whether to expect the optional values or the default values. */
+    int mode = (argc > 1) ? atoi(argv[1]) : 0;
+    
+    /* Read the data into buffer */
+    SET_BINARY_MODE(stdin);
+    count = fread(buffer, 1, sizeof(buffer), stdin);
+    
+    /* Construct a pb_istream_t for reading from the buffer */
+    stream = pb_istream_from_buffer(buffer, count);
+    
+    /* Decode and verify the message */
+    if (!check_alltypes(&stream, mode))
+    {
+        fprintf(stderr, "Test failed: %s\n", PB_GET_ERROR(&stream));
+        return 1;
+    }
+    else
+    {
+        return 0;
+    }
+}
--- a/tests/alltypes_pointer/encode_alltypes_pointer.c
+++ b/tests/alltypes_pointer/encode_alltypes_pointer.c
@@ -27,7 +27,7 @@ int main(int argc, char **argv)
    int64_t     req_sfixed64      = -1012;
    double      req_double        = 1013.0;
    char*       req_string        = "1014";
-    pb_bytes_ptr_t req_bytes      = {4, (uint8_t*)"1015"};
+    PB_BYTES_ARRAY_T(4) req_bytes = {4, {'1', '0', '1', '5'}};
    static int32_t req_substuff   = 1016;
    SubMessage  req_submsg        = {"1016", &req_substuff};
    MyEnum      req_enum          = MyEnum_Truth;
@@ -50,7 +50,8 @@ int main(int argc, char **argv)
    int64_t     rep_sfixed64[5]   = {0, 0, 0, 0, -2012};
    double      rep_double[5]     = {0, 0, 0, 0, 2013.0f};
    char*       rep_string[5]     = {"", "", "", "", "2014"};
-    pb_bytes_ptr_t rep_bytes[5]   = {{0,0}, {0,0}, {0,0}, {0,0}, {4, (uint8_t*)"2015"}};
+    static PB_BYTES_ARRAY_T(4) rep_bytes_4 = {4, {'2', '0', '1', '5'}};
+    pb_bytes_array_t *rep_bytes[5]= {NULL, NULL, NULL, NULL, (pb_bytes_array_t*)&rep_bytes_4};
    static int32_t rep_sub2zero   = 0;
    static int32_t rep_substuff2  = 2016;
    static uint32_t rep_substuff3 = 2016;
@@ -77,7 +78,7 @@ int main(int argc, char **argv)
    int64_t     opt_sfixed64      = 3052;
    double      opt_double        = 3053.0;
    char*       opt_string        = "3054";
-    pb_bytes_ptr_t opt_bytes      = {4, (uint8_t*)"3055"};
+    PB_BYTES_ARRAY_T(4) opt_bytes = {4, {'3', '0', '5', '5'}};
    static int32_t opt_substuff   = 3056;
    SubMessage  opt_submsg        = {"3056", &opt_substuff};
    MyEnum      opt_enum          = MyEnum_Truth;
@@ -117,7 +118,7 @@ int main(int argc, char **argv)
    alltypes.req_sfixed64      = &req_sfixed64;
    alltypes.req_double        = &req_double;
    alltypes.req_string        = req_string;
-    alltypes.req_bytes         = &req_bytes;
+    alltypes.req_bytes         = (pb_bytes_array_t*)&req_bytes;
    alltypes.req_submsg        = &req_submsg;
    alltypes.req_enum          = &req_enum;
    alltypes.req_emptymsg      = &req_emptymsg;
@@ -159,7 +160,7 @@ int main(int argc, char **argv)
        alltypes.opt_sfixed64      = &opt_sfixed64;
        alltypes.opt_double        = &opt_double;
        alltypes.opt_string        = opt_string;
-        alltypes.opt_bytes         = &opt_bytes;
+        alltypes.opt_bytes         = (pb_bytes_array_t*)&opt_bytes;
        alltypes.opt_submsg        = &opt_submsg;
        alltypes.opt_enum          = &opt_enum;
        alltypes.opt_emptymsg      = &opt_emptymsg;
--- a/tests/buffer_only/SConscript
+++ b/tests/buffer_only/SConscript
@@ -4,8 +4,6 @@ Import("env")

 # Take copy of the files for custom build.
 c = Copy("$TARGET", "$SOURCE")
-env.Command("pb_encode.c", "#../pb_encode.c", c)
-env.Command("pb_decode.c", "#../pb_decode.c", c)
 env.Command("alltypes.pb.h", "$BUILD/alltypes/alltypes.pb.h", c)
 env.Command("alltypes.pb.c", "$BUILD/alltypes/alltypes.pb.c", c)
 env.Command("encode_alltypes.c", "$BUILD/alltypes/encode_alltypes.c", c)
@@ -15,9 +13,15 @@ env.Command("decode_alltypes.c", "$BUILD/alltypes/decode_alltypes.c", c)
 opts = env.Clone()
 opts.Append(CPPDEFINES = {'PB_BUFFER_ONLY': 1})

+# Build new version of core
+strict = opts.Clone()
+strict.Append(CFLAGS = strict['CORECFLAGS'])
+strict.Object("pb_decode_bufonly.o", "$NANOPB/pb_decode.c")
+strict.Object("pb_encode_bufonly.o", "$NANOPB/pb_encode.c")
+
 # Now build and run the test normally.
-enc = opts.Program(["encode_alltypes.c", "alltypes.pb.c", "pb_encode.c"])
-dec = opts.Program(["decode_alltypes.c", "alltypes.pb.c", "pb_decode.c"])
+enc = opts.Program(["encode_alltypes.c", "alltypes.pb.c", "pb_encode_bufonly.o"])
+dec = opts.Program(["decode_alltypes.c", "alltypes.pb.c", "pb_decode_bufonly.o"])

 env.RunTest(enc)
 env.RunTest([dec, "encode_alltypes.output"])
--- a/tests/encode_unittests/encode_unittests.c
+++ b/tests/encode_unittests/encode_unittests.c
@@ -281,6 +281,15 @@ int main()
        TEST(WRITES(pb_encode_delimited(&s, IntegerContainer_fields, &msg),
                    "\x09\x0A\x07\x0A\x05\x01\x02\x03\x04\x05"))
    }
+
+    {
+        IntegerContainer msg = {{5, {1,2,3,4,5}}};
+        size_t size;
+        
+        COMMENT("Test pb_get_encoded_size.")
+        TEST(pb_get_encoded_size(&size, IntegerContainer_fields, &msg) &&
+             size == 9);
+    }
    
    {
        uint8_t buffer[10];
--- a/tests/extensions/decode_extensions.c
+++ b/tests/extensions/decode_extensions.c
@@ -49,7 +49,9 @@ int main(int argc, char **argv)
    }

    /* Check that the extensions decoded properly */
+    TEST(ext1.found)
    TEST(extensionfield1 == 12345)
+    TEST(ext2.found)
    TEST(strcmp(extensionfield2.test1, "test") == 0)
    TEST(extensionfield2.test2 == 54321)
    
--- a/tests/extensions/extensions.proto
+++ b/tests/extensions/extensions.proto
@@ -1,7 +1,7 @@
 import 'alltypes.proto';

 extend AllTypes {
-    optional int32 AllTypes_extensionfield1 = 255;
+    optional int32 AllTypes_extensionfield1 = 255 [default = 5];
 }

 message ExtensionMessage {
--- a/tests/field_size_16/SConscript
+++ b/tests/field_size_16/SConscript
@@ -5,8 +5,6 @@ Import("env")

 # Take copy of the files for custom build.
 c = Copy("$TARGET", "$SOURCE")
-env.Command("pb_encode.c", "#../pb_encode.c", c)
-env.Command("pb_decode.c", "#../pb_decode.c", c)
 env.Command("encode_alltypes.c", "$BUILD/alltypes/encode_alltypes.c", c)
 env.Command("decode_alltypes.c", "$BUILD/alltypes/decode_alltypes.c", c)

@@ -16,9 +14,15 @@ env.NanopbProto(["alltypes", "alltypes.options"])
 opts = env.Clone()
 opts.Append(CPPDEFINES = {'PB_FIELD_16BIT': 1})

+# Build new version of core
+strict = opts.Clone()
+strict.Append(CFLAGS = strict['CORECFLAGS'])
+strict.Object("pb_decode_fields16.o", "$NANOPB/pb_decode.c")
+strict.Object("pb_encode_fields16.o", "$NANOPB/pb_encode.c")
+
 # Now build and run the test normally.
-enc = opts.Program(["encode_alltypes.c", "alltypes.pb.c", "pb_encode.c"])
-dec = opts.Program(["decode_alltypes.c", "alltypes.pb.c", "pb_decode.c"])
+enc = opts.Program(["encode_alltypes.c", "alltypes.pb.c", "pb_encode_fields16.o"])
+dec = opts.Program(["decode_alltypes.c", "alltypes.pb.c", "pb_decode_fields16.o"])

 env.RunTest(enc)
 env.RunTest([dec, "encode_alltypes.output"])
--- a/tests/field_size_16/alltypes.proto
+++ b/tests/field_size_16/alltypes.proto
@@ -107,5 +107,7 @@ message AllTypes {
    // Just to make sure that the size of the fields has been calculated
    // properly, i.e. otherwise a bug in last field might not be detected.
    required int32      end = 10099;
+
+    extensions 200 to 255;
 }

--- a/tests/field_size_32/SConscript
+++ b/tests/field_size_32/SConscript
@@ -5,8 +5,6 @@ Import("env")

 # Take copy of the files for custom build.
 c = Copy("$TARGET", "$SOURCE")
-env.Command("pb_encode.c", "#../pb_encode.c", c)
-env.Command("pb_decode.c", "#../pb_decode.c", c)
 env.Command("encode_alltypes.c", "$BUILD/alltypes/encode_alltypes.c", c)
 env.Command("decode_alltypes.c", "$BUILD/alltypes/decode_alltypes.c", c)

@@ -16,9 +14,15 @@ env.NanopbProto(["alltypes", "alltypes.options"])
 opts = env.Clone()
 opts.Append(CPPDEFINES = {'PB_FIELD_32BIT': 1})

+# Build new version of core
+strict = opts.Clone()
+strict.Append(CFLAGS = strict['CORECFLAGS'])
+strict.Object("pb_decode_fields32.o", "$NANOPB/pb_decode.c")
+strict.Object("pb_encode_fields32.o", "$NANOPB/pb_encode.c")
+
 # Now build and run the test normally.
-enc = opts.Program(["encode_alltypes.c", "alltypes.pb.c", "pb_encode.c"])
-dec = opts.Program(["decode_alltypes.c", "alltypes.pb.c", "pb_decode.c"])
+enc = opts.Program(["encode_alltypes.c", "alltypes.pb.c", "pb_encode_fields32.o"])
+dec = opts.Program(["decode_alltypes.c", "alltypes.pb.c", "pb_decode_fields32.o"])

 env.RunTest(enc)
 env.RunTest([dec, "encode_alltypes.output"])
--- a/tests/field_size_32/alltypes.proto
+++ b/tests/field_size_32/alltypes.proto
@@ -107,5 +107,7 @@ message AllTypes {
    // Just to make sure that the size of the fields has been calculated
    // properly, i.e. otherwise a bug in last field might not be detected.
    required int32      end = 13432099;
+
+    extensions 200 to 255;
 }

--- a/tests/missing_fields/missing_fields.c
+++ b/tests/missing_fields/missing_fields.c
@@ -8,6 +8,7 @@
 int main()
 {
    uint8_t buffer[512];
+    size_t size;
    
    /* Create a message with one missing field */
    {
@@ -19,12 +20,14 @@ int main()
            printf("Encode failed.\n");
            return 1;
        }
+
+        size = stream.bytes_written;
    }

    /* Test that it decodes properly if we don't require that field */
    {
        MissingField msg = {0};
-        pb_istream_t stream = pb_istream_from_buffer(buffer, sizeof(buffer));
+        pb_istream_t stream = pb_istream_from_buffer(buffer, size);
        
        if (!pb_decode(&stream, MissingField_fields, &msg))
        {
@@ -36,7 +39,7 @@ int main()
    /* Test that it does *not* decode properly if we require the field */
    {
        AllFields msg = {0};
-        pb_istream_t stream = pb_istream_from_buffer(buffer, sizeof(buffer));
+        pb_istream_t stream = pb_istream_from_buffer(buffer, size);
        
        if (pb_decode(&stream, AllFields_fields, &msg))
        {
--- a/tests/no_errmsg/SConscript
+++ b/tests/no_errmsg/SConscript
@@ -4,8 +4,6 @@ Import("env")

 # Take copy of the files for custom build.
 c = Copy("$TARGET", "$SOURCE")
-env.Command("pb_encode.c", "#../pb_encode.c", c)
-env.Command("pb_decode.c", "#../pb_decode.c", c)
 env.Command("alltypes.pb.h", "$BUILD/alltypes/alltypes.pb.h", c)
 env.Command("alltypes.pb.c", "$BUILD/alltypes/alltypes.pb.c", c)
 env.Command("encode_alltypes.c", "$BUILD/alltypes/encode_alltypes.c", c)
@@ -15,9 +13,15 @@ env.Command("decode_alltypes.c", "$BUILD/alltypes/decode_alltypes.c", c)
 opts = env.Clone()
 opts.Append(CPPDEFINES = {'PB_NO_ERRMSG': 1})

+# Build new version of core
+strict = opts.Clone()
+strict.Append(CFLAGS = strict['CORECFLAGS'])
+strict.Object("pb_decode_noerr.o", "$NANOPB/pb_decode.c")
+strict.Object("pb_encode_noerr.o", "$NANOPB/pb_encode.c")
+
 # Now build and run the test normally.
-enc = opts.Program(["encode_alltypes.c", "alltypes.pb.c", "pb_encode.c"])
-dec = opts.Program(["decode_alltypes.c", "alltypes.pb.c", "pb_decode.c"])
+enc = opts.Program(["encode_alltypes.c", "alltypes.pb.c", "pb_encode_noerr.o"])
+dec = opts.Program(["decode_alltypes.c", "alltypes.pb.c", "pb_decode_noerr.o"])

 env.RunTest(enc)
 env.RunTest([dec, "encode_alltypes.output"])
--- a/tests/package_name/SConscript
+++ b/tests/package_name/SConscript
@@ -0,0 +1,36 @@
+# Check that alltypes test case works also when the .proto file defines
+# a package name.
+
+Import("env")
+
+# Build a modified alltypes.proto
+def modify_proto(target, source, env):
+    '''Add a "package test.package;" directive to the beginning of the .proto file.'''
+    data = open(str(source[0]), 'r').read()
+    open(str(target[0]), 'w').write("package test.package;\n\n" + data)
+    return 0
+
+env.Command("alltypes.proto", "#alltypes/alltypes.proto", modify_proto)
+env.Command("alltypes.options", "#alltypes/alltypes.options", Copy("$TARGET", "$SOURCE"))
+env.NanopbProto(["alltypes", "alltypes.options"])
+
+# Build a modified encode_alltypes.c
+def modify_c(target, source, env):
+    '''Add package name to type names in .c file.'''
+
+    data = open(str(source[0]), 'r').read()
+    
+    type_names = ['AllTypes', 'MyEnum', 'HugeEnum']
+    for name in type_names:
+        data = data.replace(name, 'test_package_' + name)
+    
+    open(str(target[0]), 'w').write(data)
+    return 0
+env.Command("encode_alltypes.c", "#alltypes/encode_alltypes.c", modify_c)
+
+# Encode and compare results to original alltypes testcase
+enc = env.Program(["encode_alltypes.c", "alltypes.pb.c", "$COMMON/pb_encode.o"])
+refdec = "$BUILD/alltypes/decode_alltypes$PROGSUFFIX"
+env.RunTest(enc)
+env.Compare(["encode_alltypes.output", "$BUILD/alltypes/encode_alltypes.output"])
+
--- a/tests/site_scons/site_init.py
+++ b/tests/site_scons/site_init.py
@@ -19,19 +19,24 @@ def add_nanopb_builders(env):
        else:
            infile = None
        
-        args = [str(source[0])]
+        if env.has_key("COMMAND"):
+            args = [env["COMMAND"]]
+        else:
+            args = [str(source[0])]
+        
        if env.has_key('ARGS'):
            args.extend(env['ARGS'])
        
+        print 'Command line: ' + str(args)
        pipe = subprocess.Popen(args,
                                stdin = infile,
                                stdout = open(str(target[0]), 'w'),
                                stderr = sys.stderr)
        result = pipe.wait()
        if result == 0:
-            print '\033[32m[ OK ]\033[0m   Ran ' + str(source[0])
+            print '\033[32m[ OK ]\033[0m   Ran ' + args[0]
        else:
-            print '\033[31m[FAIL]\033[0m   Program ' + str(source[0]) + ' returned ' + str(result)
+            print '\033[31m[FAIL]\033[0m   Program ' + args[0] + ' returned ' + str(result)
        return result
        
    run_test_builder = Builder(action = run_test,
--- a/tests/splint/SConscript
+++ b/tests/splint/SConscript
@@ -0,0 +1,13 @@
+# Check the nanopb core using splint
+
+Import('env')
+
+p = env.WhereIs('splint')
+
+if p:
+    env.Command('pb_decode.splint', '$NANOPB/pb_decode.c',
+        'splint -f splint/splint.rc $SOURCE 2> $TARGET')
+
+    env.Command('pb_encode.splint', '$NANOPB/pb_encode.c',
+        'splint -f splint/splint.rc $SOURCE 2> $TARGET')
+
--- a/tests/splint/splint.rc
+++ b/tests/splint/splint.rc
@@ -0,0 +1,37 @@
+checks
+partial
+matchanyintegral
+strictlib
+-isoreserved		# to be fixed in 0.3
+-nullassign
+-predboolint
+-predboolptr
+ptrnegate
+-switchloopbreak
+ignoresigns
+-infloopsuncon
+-type
+
+# splint's memory checks don't quite work without annotations
+-mustfreeonly
+-compmempass
+-nullret
+-observertrans
+-statictrans
+-compdestroy
+-nullpass
+-nullstate
+-compdef
+-usereleased
+-temptrans
+-dependenttrans
+-kepttrans
+-branchstate
+-immediatetrans
+
+# These tests give false positives, compiler typically has
+# better warnings for these.
+-noret
+-noeffect
+-usedef
+
Author	SHA1	Message	Date
Petteri Aimonen	2c51fb7771	Update changelog for 0.2.8	2014-05-20 19:46:48 +03:00
Petteri Aimonen	916bcb3643	Publishing nanopb-0.2.8	2014-05-20 19:35:00 +03:00
Petteri Aimonen	9cf788de54	Fix bug in alltypes test case that made fuzzing difficult.	2014-05-17 20:28:33 +03:00
Petteri Aimonen	5ef128616b	Fix security issue with PB_ENABLE_MALLOC. The multiplication in allocate_field could potentially overflow, leading to allocating too little memory. This could subsequently allow an attacker to cause a write past the buffer, overwriting other memory contents. The attack is possible if untrusted message data is decoded using nanopb, and the message type includes a pointer-type string or bytes field, or a repeated numeric field. Submessage fields are not affected. This issue only affects systems that have been compiled with PB_ENABLE_MALLOC enabled. Only version nanopb-0.2.7 is affected, as prior versions do not include this functionality. Update issue 117 Status: FixedInGit	2014-05-17 20:06:55 +03:00
Petteri Aimonen	ba2ab9ea65	Docs update, remove malloc from limitations list	2014-04-26 20:11:54 +03:00
Petteri Aimonen	e6a57e512f	Add option to not add timestamps to .pb.h and .pb.c preambles. Patch by rusnakp. Update issue 115 Status: FixedInGit	2014-04-18 15:40:40 +03:00
Petteri Aimonen	d177af1639	Fix typos in scons command line options	2014-04-15 20:30:50 +03:00
Petteri Aimonen	3b36235cef	Remove -O0 from tests CFLAGS so that optimized builds can be tested also	2014-04-15 20:27:38 +03:00
Petteri Aimonen	1d249a48ea	Fix bug in missing_fields test case	2014-04-09 19:39:12 +03:00
Petteri Aimonen	3e83d81b09	Use -fsanitize=undefined when running tests with clang	2014-04-09 19:28:57 +03:00
Petteri Aimonen	938c7ac3f3	Setting version to 0.2.8-dev	2014-04-07 20:45:04 +03:00
Petteri Aimonen	6d74c66ada	Publishing nanopb-0.2.7	2014-04-07 20:30:42 +03:00
Petteri Aimonen	c998ffe117	Update changelog	2014-04-07 20:30:12 +03:00
Petteri Aimonen	a8de6acf2d	Add rule for building coverage summary using lcov. Also modified a few tests to be more compatible with coverage information, so that they use the same pb_encode/decode.c instead of making a copy.	2014-04-05 13:26:37 +03:00
Petteri Aimonen	7880f308ea	Fix unused parameter warning when building without errmsg.	2014-04-05 13:25:44 +03:00
Petteri Aimonen	b63e582bdb	Add a convenience function pb_get_encoded_size() There is minimal size penalty from this, and it is probably much more intuitive to use than PB_OSTREAM_SIZING itself. This has been suggested before also, but I ended up refusing it back them. Reconsidering it now, I see that an intuitive API is much better than any amount of documentation explaining a non-intuitive API. Update issue 16 Status: FixedInGit	2014-04-05 11:26:39 +03:00
Petteri Aimonen	e5b855fec5	Add a 'found' field to pb_extension_t. Update issue 112 Status: FixedInGit	2014-04-05 11:11:05 +03:00
Petteri Aimonen	70dee34da6	Add some missing 'static' specifiers Update issue 91 Status: FixedInGit	2014-04-02 21:08:15 +03:00
Petteri Aimonen	99434724d0	Fix splint warnings, add splint test case	2014-04-02 21:07:30 +03:00
Petteri Aimonen	6c90e824c4	Fix compile error when default value given for extension field. Update issue 111 Status: FixedInGit	2014-04-01 16:47:53 +03:00
Petteri Aimonen	f4949119ad	Add stdlib.h to pb_syshdr.h for dynamic allocation	2014-03-18 16:13:54 +02:00
Petteri Aimonen	607cb998b5	More configuration options for dynamic alloc	2014-03-17 17:25:58 +02:00
Petteri Aimonen	ab62402059	Documentation updates	2014-03-16 15:52:19 +02:00
Petteri Aimonen	108864963f	Finish the alltypes_pointer testcase, use valgrind if available.	2014-03-15 09:39:27 +02:00
Petteri Aimonen	9be2cfe968	Get rid of pb_bytes_ptr_t, just allocate pb_bytes_array_t dynamically. This makes the internal logic much simpler, and also keeps the datatypes more similar between STATIC/POINTER cases. It will still be a bit cumbersome to use because of variable length array member. Macros PB_BYTES_ARRAY_T(n) and PB_BYTES_ARRAY_T_ALLOCSIZE(n) have been added to make life a bit easier. This has the drawback that it is no longer as easy to use externally allocated byte array as input for bytes field in pointer mode. However, this is still easy to do using callbacks, so it shouldn't be a large issue.	2014-03-15 08:45:58 +02:00
Petteri Aimonen	9c196b89ba	Add pb_release() function	2014-03-12 21:08:35 +02:00
Petteri Aimonen	bf61d2337b	More fixes for dynamic allocation	2014-03-10 18:19:38 +02:00
Petteri Aimonen	48ac461372	Bugfixes for dynamic allocation	2014-02-25 19:58:11 +02:00
Petteri Aimonen	011a30af9c	Beginnings of malloc support in pb_decode	2014-02-24 21:09:25 +02:00
Petteri Aimonen	842c960d5d	Setting version to 0.2.7-dev	2014-02-15 17:15:54 +02:00
Petteri Aimonen	7f397b067f	Publishing nanopb-0.2.6	2014-02-15 16:40:38 +02:00
Petteri Aimonen	586777b52f	Update changelog	2014-02-12 19:50:52 +02:00
Petteri Aimonen	542463dbaa	Move descriptor.proto to google/protobuf directory where it belongs. This fixes some interoperability problems when trying to compile nanopb.proto for e.g. the Google's C++ library. Update issue 104 Status: FixedInGit	2014-02-12 19:43:04 +02:00
Petteri Aimonen	fe0bf121eb	Add test case for package names defined in .proto file.	2014-02-12 19:22:01 +02:00
Petteri Aimonen	bd22cf2777	Improve status/error reporting in generator. Update issue 105 Status: FixedInGit	2014-02-12 18:51:48 +02:00
Petteri Aimonen	057165966c	Add AllTypes decoding test with callbacks. Update issue 99 Status: FixedInGit	2014-02-10 21:00:26 +02:00
Petteri Aimonen	235219a295	Add AllTypes encoding test for callbacks (issue #99 ) Still need to do the decoding side.	2014-02-09 19:28:50 +02:00
Petteri Aimonen	d3ed0744d0	Extreme negative integer constants are a bit nasty to define in C, just avoid the trouble in tests for now.	2014-02-05 17:56:37 +02:00
Petteri Aimonen	879860be19	Add coments to places where STATIC_ASSERT is used. Update issue 96 Status: FixedInGit	2014-02-04 20:34:57 +02:00
Petteri Aimonen	a2f8112166	Add proper suffixes for integer default values. Update issue 102 Status: FixedInGit	2014-02-04 20:24:44 +02:00
Petteri Aimonen	5efeb392e0	Add strict-aliasing GCC warnings for core	2014-02-04 20:11:52 +02:00
Petteri Aimonen	a46ed9f475	Fix problem with callback-type fields (issue #99 ).	2014-01-03 11:17:39 +02:00
Petteri Aimonen	843fc4b11a	Setting version to 0.2.6-dev	2014-01-01 11:44:47 +02:00