Merge branch 'develop'

.gitignore

@@ -4,6 +4,10 @@
 *.swp
 *.gcov
 /.sconsign.dblite
+/.sconf_temp/
+/config.log
+mpack-test-file
+/mpack-test-dir/
 
 # folders
 /tags

.travis.yml

@@ -1,5 +1,8 @@
 language: c
 
+before_install:
+  - pip install --user cpp-coveralls
+
 addons:
   apt:
     packages:

Doxyfile

@@ -40,6 +40,8 @@ PREDEFINED = \
     MPACK_READ_TRACKING=1 \
     MPACK_WRITE_TRACKING=1 \
     \
+    MPACK_HEADER_START= \
+    MPACK_HEADER_END= \
 
 MARKDOWN_SUPPORT = YES
 JAVADOC_AUTOBRIEF = YES

README.md

@@ -1,5 +1,3 @@
-[![Build Status](https://travis-ci.org/ludocode/mpack.svg?branch=master)](https://travis-ci.org/ludocode/mpack/branches)
-
 ## Introduction
 
 MPack is a C implementation of an encoder and decoder for the [MessagePack](http://msgpack.org/) serialization format. It is intended to be:
@@ -16,7 +14,13 @@ The MPack code is small enough to be embedded directly into your codebase. The e
 
 MPack is written in the portable intersection of C99 and C++. In other words, it's written in C99, but if you are stuck using a certain popular compiler from a certain unpopular vendor that refuses to support C99, you can compile it as C++ instead.
 
-*NOTE: MPack is beta software under development. The API occasionally changes, there are still some TODOs in the codebase, some security issues to fix, some MessagePack 1.0/1.1 compatibility and interoperability issues to sort out, some test suite portability issues to fix, and there is only around 65% unit test coverage.*
+## Build Status
+
+MPack is beta software under development.
+
+| [Travis-CI](https://travis-ci.org/) | [Coveralls.io](https://coveralls.io/) |
+| :-------: | :----------: |
+| [![Build Status](https://travis-ci.org/ludocode/mpack.svg?branch=master)](https://travis-ci.org/ludocode/mpack/branches) | [![Coverage Status](https://coveralls.io/repos/ludocode/mpack/badge.svg?branch=master&service=github)](https://coveralls.io/github/ludocode/mpack?branch=master) |
 
 ## The Node Reader API
 
@@ -39,7 +43,7 @@ if (mpack_tree_destroy(tree) != mpack_ok) {
 }
 ```
 
-Note that no additional error handling is needed in the above code. If the file is missing or corrupt, if map keys are missing or if nodes are not in the expected types, special "nil" nodes and false/zero values are returned and the tree is placed in an error state. An error check is only needed before using the data. Alternatively, the tree can be configured to longjmp in such cases if a handler is set.
+Note that no additional error handling is needed in the above code. If the file is missing or corrupt, if map keys are missing or if nodes are not in the expected types, special "nil" nodes and false/zero values are returned and the tree is placed in an error state. An error check is only needed before using the data.
 
 ## The Write API
 
@@ -73,7 +77,7 @@ free(data);
 
 In the above example, we encode to a growable memory buffer. The writer can instead write to a pre-allocated or stack-allocated buffer, avoiding the need for memory allocation. The writer can also be provided with a flush function (such as a file or socket write function) to call when the buffer is full or when writing is done.
 
-If any error occurs, the writer is placed in an error state and can optionally longjmp if a handler is set. The writer will flag an error if too much data is written, if the wrong number of elements are written, if the data could not be flushed, etc. No additional error handling is needed in the above code; any subsequent writes are ignored when the writer is in an error state, so you don't need to check every write for errors.
+If any error occurs, the writer is placed in an error state. The writer will flag an error if too much data is written, if the wrong number of elements are written, if the data could not be flushed, etc. No additional error handling is needed in the above code; any subsequent writes are ignored when the writer is in an error state, so you don't need to check every write for errors.
 
 Note in particular that in debug mode, the `mpack_finish_map()` call above ensures that two key/value pairs were actually written as claimed, something that other MessagePack C/C++ libraries may not do.
 
@@ -83,7 +87,7 @@ Conceptually, MessagePack stores data similarly to JSON: they are both composed
 
 - Compound types such as strings, maps and arrays are delimited, so appropriate storage cannot be allocated upfront. The whole object must be parsed to determine its size.
 
-- Strings are not stored in their native encoding. They cannot contain quotes or special characters, so they must be escaped when written and converted back when read.
+- Strings are not stored in their native encoding. Special characters such as quotes and backslashes must be escaped when written and converted back when read.
 
 - Numbers are particularly inefficient (especially when parsing back floats), making JSON inappropriate as a base format for structured data that contains lots of numbers.
 

SConscript

@@ -2,6 +2,15 @@ import platform
 
 Import('env', 'CPPFLAGS', 'LINKFLAGS')
 
+# we add the C/C++ specific flags here. we can't use CCFLAGS/CXXFLAGS
+# because as far as SCons is concerned, they are all C files; we're
+# passing -x c++ to force the language.
+if "c++" in CPPFLAGS:
+    CPPFLAGS += ["-Wmissing-declarations"]
+    LINKFLAGS += ["-lstdc++"]
+else:
+    CPPFLAGS += ["-Wmissing-prototypes", "-Wc++-compat"]
+
 srcs = env.Object(env.Glob('src/mpack/*.c') + env.Glob('test/*.c'),
         CPPFLAGS=env['CPPFLAGS'] + CPPFLAGS)
 

SConstruct

@@ -1,13 +1,26 @@
 import platform, os
 
+def CheckFlags(context, cppflags, linkflags = [], message = None):
+    if message == None:
+        message = " ".join(cppflags + ((cppflags != linkflags) and linkflags or []))
+    context.Message("Checking for " + message + " support... ")
+
+    env.Prepend(CPPFLAGS = cppflags, LINKFLAGS = linkflags)
+    result = context.TryLink("int main(void){return 0;}\n", '.c')
+    env.Replace(CPPFLAGS = env["CPPFLAGS"][len(cppflags):], LINKFLAGS = env["LINKFLAGS"][len(linkflags):])
+
+    context.Result(result)
+    return result
+
 
 # Common environment setup
 
 env = Environment()
+conf = Configure(env, custom_tests = {'CheckFlags': CheckFlags})
+
 for x in os.environ.keys():
     if x in ["CC", "CXX", "PATH", "TRAVIS", "TERM"] or x.startswith("CLANG_") or x.startswith("CCC_"):
         env[x] = os.environ[x]
-        env["ENV"][x] = os.environ[x]
 
 env.Append(CPPFLAGS = [
     "-Wall", "-Wextra", "-Werror",
@@ -19,11 +32,12 @@ env.Append(CPPFLAGS = [
 env.Append(LINKFLAGS = [
     "-g",
     ])
+# Additional warning flags are passed in SConscript based on the language (C/C++)
 
-if 'CC' not in env or "clang" not in env['CC']:
-    env.Append(CPPFLAGS = ["-Wno-float-conversion"]) # unsupported in clang 3.4
-    env.Append(CPPFLAGS = ["-fprofile-arcs", "-ftest-coverage"]) # only works properly with gcc
-    env.Append(LINKFLAGS = ["-fprofile-arcs", "-ftest-coverage"])
+if 'TRAVIS' not in env:
+    # Travis-CI currently uses Clang 3.4 which does not support this option,
+    # and it also appears to be incompatible with other GCC options on Travis-CI
+    env.Append(CPPFLAGS = ["-Wno-float-conversion"])
 
 
 # Optional flags used in various builds
@@ -41,74 +55,104 @@ noioconfigs = [
 ]
 allconfigs = noioconfigs + ["-DMPACK_STDIO=1"]
 
-debugflags = ["-DDEBUG", "-O0"]
-releaseflags = ["-Os"] # If you change this, also change the MPACK_OPTIMIZE_FOR_SIZE below to test the opposite
-cflags = ["-std=c99", "-Wc++-compat"]
+hasOg = conf.CheckFlags(["-Og"])
+if hasOg:
+    debugflags = ["-DDEBUG", "-Og"]
+else:
+    debugflags = ["-DDEBUG", "-O0"]
+releaseflags = ["-Os"]
+cflags = ["-std=c99"]
+
+gcovflags = []
+if ARGUMENTS.get('gcov'):
+    gcovflags = ["-DMPACK_GCOV=1", "--coverage"]
+
+ltoflags = ["-O3", "-flto", "-fuse-linker-plugin", "-fno-fat-lto-objects"]
+
+# -lstdc++ is added in SConscript
+cxxflags = ["-x", "c++"]
 
 
 # Functions to add a variant build. One variant build will build and run the
 # entire library and test suite in a given configuration.
 
-def AddBuild(variant_dir, cppflags, linkflags):
+def AddBuild(variant_dir, cppflags, linkflags = []):
     env.SConscript("SConscript",
             variant_dir="build/" + variant_dir,
             src="../..",
-            exports={'env': env, 'CPPFLAGS': cppflags, 'LINKFLAGS': linkflags},
+            exports={
+                'env': env,
+                'CPPFLAGS': cppflags,
+                'LINKFLAGS': linkflags
+                },
             duplicate=0)
 
-def AddBuilds(variant_dir, cppflags, linkflags):
+def AddBuilds(variant_dir, cppflags, linkflags = []):
     AddBuild("debug-" + variant_dir, debugflags + cppflags, debugflags + linkflags)
-    AddBuild("release-" + variant_dir, releaseflags + cppflags, releaseflags + linkflags)
+    if ARGUMENTS.get('all'):
+        AddBuild("release-" + variant_dir, releaseflags + cppflags, releaseflags + linkflags)
 
 
-# The default build, everything in debug. Run "scons dev=1" during
-# development to build only this. You would also use this for static
-# analysis, e.g. "scan-build scons dev=1".
+# The default build, everything in debug. This is the build used
+# for code coverage measurement and static analysis.
 
-AddBuild("debug", allfeatures + allconfigs + debugflags + cflags, [])
+AddBuild("debug", allfeatures + allconfigs + debugflags + cflags + gcovflags, gcovflags)
 
 
-# Otherwise run all builds. Use a parallel build, e.g. "scons -j12" to
-# build this in a reasonable amount of time.
+# Run "scons more=1" to run a handful of builds that are likely
+# to reveal configuration errors.
+if ARGUMENTS.get('more') or ARGUMENTS.get('all'):
+    AddBuild("release", allfeatures + allconfigs + releaseflags + cflags)
+    AddBuilds("embed", allfeatures + cflags)
+    AddBuilds("noio", allfeatures + noioconfigs + cflags)
 
+
+# Run "scons all=1" to run all builds. This is what the CI runs.
 if ARGUMENTS.get('all'):
-    AddBuild("release", allfeatures + allconfigs + releaseflags + cflags, [])
+
+    # various release builds
+    AddBuild("release-unopt", allfeatures + allconfigs + cflags + ["-O0"])
+    AddBuild("release-fastmath", allfeatures + allconfigs + releaseflags + cflags + ["-ffast-math"])
     AddBuild("release-speed", ["-DMPACK_OPTIMIZE_FOR_SIZE=0"] +
-            allfeatures + allconfigs + releaseflags + cflags, [])
+            allfeatures + allconfigs + releaseflags + cflags)
+    if conf.CheckFlags(ltoflags, ltoflags, "-flto"):
+        AddBuild("release-lto", allfeatures + allconfigs + ltoflags + cflags, ltoflags)
 
     # feature subsets with default configuration
-    AddBuilds("empty", allconfigs + cflags, [])
-    AddBuilds("writer", ["-DMPACK_WRITER=1"] + allconfigs + cflags, [])
-    AddBuilds("reader", ["-DMPACK_READER=1"] + allconfigs + cflags, [])
-    AddBuilds("expect", ["-DMPACK_READER=1", "-DMPACK_EXPECT=1"] + allconfigs + cflags, [])
-    AddBuilds("node", ["-DMPACK_NODE=1"] + allconfigs + cflags, [])
+    AddBuilds("empty", allconfigs + cflags)
+    AddBuilds("writer", ["-DMPACK_WRITER=1"] + allconfigs + cflags)
+    AddBuilds("reader", ["-DMPACK_READER=1"] + allconfigs + cflags)
+    AddBuilds("expect", ["-DMPACK_READER=1", "-DMPACK_EXPECT=1"] + allconfigs + cflags)
+    AddBuilds("node", ["-DMPACK_NODE=1"] + allconfigs + cflags)
 
     # no i/o
-    AddBuilds("noio", allfeatures + noioconfigs + cflags, [])
-    AddBuilds("noio-writer", ["-DMPACK_WRITER=1"] + noioconfigs + cflags, [])
-    AddBuilds("noio-reader", ["-DMPACK_READER=1"] + noioconfigs + cflags, [])
-    AddBuilds("noio-expect", ["-DMPACK_READER=1", "-DMPACK_EXPECT=1"] + noioconfigs + cflags, [])
-    AddBuilds("noio-node", ["-DMPACK_NODE=1"] + noioconfigs + cflags, [])
+    AddBuilds("noio-writer", ["-DMPACK_WRITER=1"] + noioconfigs + cflags)
+    AddBuilds("noio-reader", ["-DMPACK_READER=1"] + noioconfigs + cflags)
+    AddBuilds("noio-expect", ["-DMPACK_READER=1", "-DMPACK_EXPECT=1"] + noioconfigs + cflags)
+    AddBuilds("noio-node", ["-DMPACK_NODE=1"] + noioconfigs + cflags)
 
     # embedded builds without libc
-    AddBuilds("embed", allfeatures + cflags, [])
-    AddBuilds("embed-writer", ["-DMPACK_WRITER=1"] + cflags, [])
-    AddBuilds("embed-reader", ["-DMPACK_READER=1"] + cflags, [])
-    AddBuilds("embed-expect", ["-DMPACK_READER=1", "-DMPACK_EXPECT=1"] + cflags, [])
-    AddBuilds("embed-node", ["-DMPACK_NODE=1"] + cflags, [])
+    AddBuilds("embed-writer", ["-DMPACK_WRITER=1"] + cflags)
+    AddBuilds("embed-reader", ["-DMPACK_READER=1"] + cflags)
+    AddBuilds("embed-expect", ["-DMPACK_READER=1", "-DMPACK_EXPECT=1"] + cflags)
+    AddBuilds("embed-node", ["-DMPACK_NODE=1"] + cflags)
 
     # miscellaneous test builds
-    AddBuilds("cxx", allfeatures + allconfigs + ["-xc++", "-std=c++98"], ["-lstdc++"])
-    AddBuilds("notrack", ["-DMPACK_NO_TRACKING=1"] + allfeatures + allconfigs + cflags, [])
-    AddBuilds("realloc", allfeatures + allconfigs + debugflags + cflags + ["-DMPACK_REALLOC=test_realloc"], [])
-
-    # Travis-CI currently only has GCC 4.6 and Clang 3.4, doesn't properly support these options
-    if 'TRAVIS' not in env:
-        AddBuilds("c11", allfeatures + allconfigs + ["-std=c11", "-Wc++-compat"], [])
-        AddBuilds("cxx11", allfeatures + allconfigs + ["-xc++", "-std=c++11"], ["-lstdc++"])
-        AddBuilds("cxx14", allfeatures + allconfigs + ["-xc++", "-std=c++14"], ["-lstdc++"])
-
-    # 32-bit builds (Travis-CI doesn't support multilib)
-    if 'TRAVIS' not in env and '64' in platform.architecture()[0]:
+    AddBuilds("notrack", ["-DMPACK_NO_TRACKING=1"] + allfeatures + allconfigs + cflags)
+    AddBuilds("realloc", allfeatures + allconfigs + debugflags + cflags + ["-DMPACK_REALLOC=test_realloc"])
+    if hasOg:
+        AddBuild("debug-O0", allfeatures + allconfigs + ["-DDEBUG", "-O0"] + cflags)
+
+    # other language standards (C11, various C++ versions)
+    if conf.CheckFlags(["-std=c11"]):
+        AddBuilds("c11", allfeatures + allconfigs + ["-std=c11"])
+    AddBuilds("cxx", allfeatures + allconfigs + cxxflags + ["-std=c++98"])
+    if conf.CheckFlags(cxxflags + ["-std=c++11"], [], "-std=c++11"):
+        AddBuilds("cxx11", allfeatures + allconfigs + cxxflags + ["-std=c++11"])
+    if conf.CheckFlags(cxxflags + ["-std=c++14"], [], "-std=c++14"):
+        AddBuilds("cxx14", allfeatures + allconfigs + cxxflags + ["-std=c++14"])
+
+    # 32-bit build
+    if conf.CheckFlags(["-m32"], ["-m32"]):
         AddBuilds("32",     allfeatures + allconfigs + cflags + ["-m32"], ["-m32"])
-        AddBuilds("cxx11-32", allfeatures + allconfigs + ["-xc++", "-std=c++11"] + ["-m32"], ["-m32", "-lstdc++"])
+        AddBuilds("cxx32",  allfeatures + allconfigs + cxxflags + ["-std=c++98", "-m32"], ["-m32"])

projects/vs/mpack.vcxproj

@@ -81,13 +81,14 @@
     <ClCompile Include="..\..\src\mpack\mpack-platform.c" />
     <ClCompile Include="..\..\src\mpack\mpack-reader.c" />
     <ClCompile Include="..\..\src\mpack\mpack-writer.c" />
-    <ClCompile Include="..\..\test\main.c" />
+    <ClCompile Include="..\..\test\test-reader.c" />
+    <ClCompile Include="..\..\test\test.c" />
     <ClCompile Include="..\..\test\test-buffer.c" />
     <ClCompile Include="..\..\test\test-file.c" />
-    <ClCompile Include="..\..\test\test-malloc.c" />
+    <ClCompile Include="..\..\test\test-system.c" />
     <ClCompile Include="..\..\test\test-node.c" />
     <ClCompile Include="..\..\test\test-expect.c" />
-    <ClCompile Include="..\..\test\test-tag.c" />
+    <ClCompile Include="..\..\test\test-common.c" />
     <ClCompile Include="..\..\test\test-write.c" />
   </ItemGroup>
   <ItemGroup>
@@ -101,14 +102,15 @@
     <ClInclude Include="..\..\test\mpack-config.h" />
     <ClInclude Include="..\..\test\test-buffer.h" />
     <ClInclude Include="..\..\test\test-file.h" />
-    <ClInclude Include="..\..\test\test-malloc.h" />
+    <ClInclude Include="..\..\test\test-reader.h" />
+    <ClInclude Include="..\..\test\test-system.h" />
     <ClInclude Include="..\..\test\test-node.h" />
     <ClInclude Include="..\..\test\test-expect.h" />
-    <ClInclude Include="..\..\test\test-tag.h" />
+    <ClInclude Include="..\..\test\test-common.h" />
     <ClInclude Include="..\..\test\test-write.h" />
     <ClInclude Include="..\..\test\test.h" />
   </ItemGroup>
   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
   <ImportGroup Label="ExtensionTargets">
   </ImportGroup>
-</Project>
+</Project>

projects/vs/mpack.vcxproj.filters

@@ -29,7 +29,7 @@
     <ClCompile Include="..\..\src\mpack\mpack-writer.c">
       <Filter>Source Files</Filter>
     </ClCompile>
-    <ClCompile Include="..\..\test\main.c">
+    <ClCompile Include="..\..\test\test.c">
       <Filter>Source Files</Filter>
     </ClCompile>
     <ClCompile Include="..\..\test\test-buffer.c">
@@ -38,7 +38,7 @@
     <ClCompile Include="..\..\test\test-file.c">
       <Filter>Source Files</Filter>
     </ClCompile>
-    <ClCompile Include="..\..\test\test-malloc.c">
+    <ClCompile Include="..\..\test\test-system.c">
       <Filter>Source Files</Filter>
     </ClCompile>
     <ClCompile Include="..\..\test\test-node.c">
@@ -47,12 +47,15 @@
     <ClCompile Include="..\..\test\test-expect.c">
       <Filter>Source Files</Filter>
     </ClCompile>
-    <ClCompile Include="..\..\test\test-tag.c">
+    <ClCompile Include="..\..\test\test-common.c">
       <Filter>Source Files</Filter>
     </ClCompile>
     <ClCompile Include="..\..\test\test-write.c">
       <Filter>Source Files</Filter>
     </ClCompile>
+    <ClCompile Include="..\..\test\test-reader.c">
+      <Filter>Source Files</Filter>
+    </ClCompile>
   </ItemGroup>
   <ItemGroup>
     <ClInclude Include="..\..\src\mpack\mpack-common.h">
@@ -85,7 +88,7 @@
     <ClInclude Include="..\..\test\test-file.h">
       <Filter>Header Files</Filter>
     </ClInclude>
-    <ClInclude Include="..\..\test\test-malloc.h">
+    <ClInclude Include="..\..\test\test-system.h">
       <Filter>Header Files</Filter>
     </ClInclude>
     <ClInclude Include="..\..\test\test-node.h">
@@ -94,7 +97,7 @@
     <ClInclude Include="..\..\test\test-expect.h">
       <Filter>Header Files</Filter>
     </ClInclude>
-    <ClInclude Include="..\..\test\test-tag.h">
+    <ClInclude Include="..\..\test\test-common.h">
       <Filter>Header Files</Filter>
     </ClInclude>
     <ClInclude Include="..\..\test\test-write.h">
@@ -103,5 +106,8 @@
     <ClInclude Include="..\..\test\test.h">
       <Filter>Header Files</Filter>
     </ClInclude>
+    <ClInclude Include="..\..\test\test-reader.h">
+      <Filter>Header Files</Filter>
+    </ClInclude>
   </ItemGroup>
-</Project>
+</Project>