packages feed

protocol-buffers 1.2.2 → 1.4.0

raw patch · 13 files changed

+332/−490 lines, 13 files

Files

README view
@@ -1,6 +1,6 @@ This the README file for protocol-buffers, protocol-buffers-descriptors, and hprotoc. These are three interdependent Haskell packages by Chris Kuklewicz.-This README was updated most recently to reflect version 0.3.1+This README was updated most recently to reflect version 1.4.0  Questions and answers: @@ -15,7 +15,7 @@    This provides non-mutable messages that ought to be wire-compatible with Google.   These messages support extensions.-  These messages support unknown fields if hprotoc is passed the proper flag.+  These messages support unknown fields if hprotoc is passed the proper flag (-u or --unknown_fields).   This does not generate anything for Services/Methods.    Adding support for services has not been considered.@@ -36,6 +36,7 @@   1) It uses the protocol-buffers package.   2) It provides the code generated by hprotoc from "descriptor.proto" under module Text.DescriptorProtos.   3) This supports hprotoc which is used to describe proto files and the code they will generate.+  4) It provides Text.DescriptorProtos.Options which help in looking up the new style custom options.  What is hprotoc? @@ -51,13 +52,13 @@    The examples sub-directory is for duplicating the addressbook.proto example that Google has with its code.  The ABF and ABF2 file are included as binary addressbooks.  These can be read by the C++ examples from Google, and vice-versa. -  The tests sub-directory is where I have written some test code to drive the UnittestProto code generated from Google's unittest.proto (and unittest_import.proto) files.  The 'patchBoot' file has the needed file patches to fix up the recursive imports.+  The tests sub-directory is where I have written some test code to drive the UnittestProto code generated from Google's unittest.proto (and unittest_import.proto) files.  The 'patchBoot' file has the needed file patches to fix up the recursive imports (no longer needed!).  What do I need to compile the code? -  I use ghc (version 6.8.3) and cabal (version 1.2.4.0).+  I use ghc (version 6.10.1, previously version 6.8.3) and cabal (version 1.6.0.1, previously version 1.2.4.0). -  The dependencies are listed in the .cabal files, and these currently require you to go to hackage.haskell.org and get packages "binary" (I use version 0.4.2) and "utf8-string" (I use 0.3.1.1).+  The dependencies are listed in the .cabal files, and these currently require you to go to hackage.haskell.org and get packages "binary" (I use version 0.4.4, previously version 0.4.2) and "utf8-string" (I use version 0.3.3, previously version 0.3.1.1) and hprotoc needs haskell-src-exts (exactly the "latest" version 4.8.0, upgrades tend to break compilation until (easily) patched).    The hprotoc Lexer.hs is produced from Lexer.x by the alex program (I use version 2.2) which can be downloaded from http://www.haskell.org/alex/ if you edit Lexer.x and need to regenerate Lexer.hs. @@ -83,9 +84,7 @@  Mutual recursion is a problem? -  It is not fully automatic with GHC.  One can start with the Skeleton.hs-boot file (an extra in the source tree) or just write the needed hs-boot files by hand to break the recursion.  This is very similar to the making header files in C with forward declarations.  See the GHC user manual for more.--  Perhaps a future version will generate the needed hs-boot files automatically.  But the effort of add that feature is more than effort of making the hs-boot files manually.+  Not using ghc.  The haskell-src-exts let me generate code with {-# SOURCE #-} annotated imports.  And hprotoc generates the needed hs-boot files for ghc.  And key import cycles are broken by creating 'Key.hs files, which users can ignore.  How stable is the API? @@ -145,6 +144,8 @@     This stage also determines that extension keys are in a valid extensions range declaration, and enum default values exists.   The MakeReflections.hs file converts the nested FileDescriptorProto into a flatter Haskell reflection data structure.     This includes parsing the default value stored in the FileDescriptorProto.+  The BreakRecursion.hs file builds graphs describing the imports and works out whether and how to create hs-boot and 'Key.hs files+    to allow allow for warning-free compilation with ghc (as of 6.10.1).   The Gen.hs file takes a Haskell data structure from MakeReflections and builds a module syntax data structure.     The syntax data is quite verbose and several helper functions are used to help with the composition.     The result is easy to print as a string to a file.
− Skeleton.hs-boot 
@@ -1,19 +0,0 @@-module $PARENT.$NAME ($NAME) where--import qualified Prelude as P'(Show,Eq,Ord,Maybe,Double,Float)-import qualified Text.ProtocolBuffers.Header as P'(Typeable,Mergeable,Default,Wire,MessageAPI,GPB,ReflectDescriptor-                                                  ,Seq,Utf8,ByteString,Int32,Int64,Word32,Word64,ExtendMessage)--data $NAME--instance P'.Show $NAME-instance P'.Eq $NAME-instance P'.Ord $NAME-instance P'.Typeable $NAME-instance P'.Mergeable $NAME-instance P'.Default $NAME-instance P'.Wire $NAME-instance P'.MessageAPI msg' (msg' -> $NAME) $NAME-instance P'.GPB $NAME-instance P'.ExtendMessage $NAME-instance P'.ReflectDescriptor $NAME
TODO view
@@ -1,8 +1,3 @@-Change Resolve to separate the conversion to fully qualified names from the name mangling.-  This should allow hprotoc to create FileDescriptorSet output that mimics protoc--How does protoc parse enum options and enum value options??  In the new version!- Add even more of the documentation for the public API. delete commented out code add strictness annotations to internal data types.
Text/ProtocolBuffers.hs view
@@ -5,7 +5,7 @@  @ import Text.ProtocolBuffers.Basic-  ( Seq,Utf8(utf8),Int32,Int64,Word32,Word64+  ( Seq,isValidUTF8,toUtf8,utf8,Utf8(Utf8),Int32,Int64,Word32,Word64   , WireTag,FieldId,WireType,FieldType,EnumCode,WireSize   , Mergeable(mergeEmpty,mergeAppend,mergeConcat),Default(defaultValue),Wire) import Text.ProtocolBuffers.Default()@@ -34,6 +34,11 @@ not be used as required values are filled in with undefined errors, please use 'defaultValue' instead. +The Utf8 type is a newtype of the Lazy ByteString.  It can be safely+constructed by checking for errors with 'toUtf8', which returns 'Left+Int' indicating the index where an error is detected.  It can be+deconstructed with 'utf8'.+ -} module Text.ProtocolBuffers(     module Text.ProtocolBuffers.Basic@@ -44,7 +49,7 @@   ) where  import Text.ProtocolBuffers.Basic-  ( Seq,Utf8(utf8),Int32,Int64,Word32,Word64+  ( Seq,isValidUTF8,toUtf8,utf8,Utf8(Utf8),Int32,Int64,Word32,Word64   , WireTag,FieldId,WireType,FieldType,EnumCode,WireSize   , Mergeable(mergeEmpty,mergeAppend,mergeConcat),Default(defaultValue),Wire) import Text.ProtocolBuffers.Default()
Text/ProtocolBuffers/Basic.hs view
@@ -4,12 +4,12 @@ -- 'Default', and 'Wire' classes. module Text.ProtocolBuffers.Basic   ( -- * Basic types for protocol buffer fields in Haskell-    Seq,Utf8(..),ByteString,Int32,Int64,Word32,Word64+    Seq,Utf8(Utf8),ByteString,Int32,Int64,Word32,Word64     -- * Haskell types that act in the place of DescritorProto values   , WireTag(..),FieldId(..),WireType(..),FieldType(..),EnumCode(..),WireSize     -- * Some of the type classes implemented messages and fields   , Mergeable(..),Default(..),Wire(..)-  , isValidUTF8+  , isValidUTF8, toUtf8, utf8   ) where  import Data.Binary.Put(Put)@@ -33,16 +33,17 @@ -- | 'Utf8' is used to mark 'ByteString' values that (should) contain -- valud utf8 encoded strings.  This type is used to represent -- 'TYPE_STRING' values.-newtype Utf8 = Utf8 {utf8 :: ByteString}-  deriving (Data,Typeable,Eq,Ord)+newtype Utf8 = Utf8 ByteString deriving (Data,Typeable,Eq,Ord) +utf8 :: Utf8 -> ByteString+utf8 (Utf8 bs) = bs+ instance Read Utf8 where-  readsPrec d xs =-    let r :: Int -> ReadS String-        r = readsPrec-        f :: (String,String) -> (Utf8,String)-        f (a,b) = (Utf8 (U.fromString a),b)-    in map f . r d $ xs+  readsPrec d xs = let r :: Int -> ReadS String+                       r = readsPrec+                       f :: (String,String) -> (Utf8,String)+                       f (a,b) = (Utf8 (U.fromString a),b)+                   in map f . r d $ xs  instance Show Utf8 where   showsPrec d (Utf8 bs) = let s :: Int -> String -> ShowS@@ -218,3 +219,7 @@   high (x:xs) n | 128 <= x && x <= 143 = go 2 xs $! succ n                 | otherwise = Just n   high [] n = Just n++toUtf8 :: ByteString -> Either Int Utf8+toUtf8 b = maybe (Right (Utf8 b)) Left (isValidUTF8 b)+
Text/ProtocolBuffers/Extensions.hs view
@@ -32,7 +32,7 @@ import Data.Generics import Data.Map(Map) import qualified Data.Map as M-import Data.Maybe(fromMaybe)+import Data.Maybe(fromMaybe,isJust) import Data.Monoid(mappend) import Data.Sequence(Seq,(|>)) import qualified Data.Sequence as Seq@@ -612,7 +612,7 @@  instance (Default msg,Default a) => MessageAPI msg (msg -> Maybe a) a where   getVal m f = fromMaybe (fromMaybe defaultValue (f defaultValue)) (f m)-  isSet m f = maybe False (const True) (f m)+  isSet m f = isJust (f m)  instance MessageAPI msg (msg -> (Seq a)) (Seq a) where   getVal m f = f m
Text/ProtocolBuffers/Get.hs view
@@ -58,7 +58,7 @@ -- import qualified Data.Binary.Strict.Class as P(BinaryParser(..)) -- The Get monad is an instance of all of these library classes: import Control.Applicative(Applicative(pure,(<*>)),Alternative(empty,(<|>)))-import Control.Monad(MonadPlus(mzero,mplus))+import Control.Monad(MonadPlus(mzero,mplus),when) import Control.Monad.Error.Class(MonadError(throwError,catchError),Error(strMsg)) -- It can be a MonadCont, but the semantics are too broken without a ton of work. @@ -163,7 +163,7 @@ lookAheadM todo = do   setCheckpoint   a <- todo-  maybe useCheckpoint (\_ -> clearCheckpoint) a+  maybe useCheckpoint (const clearCheckpoint) a   return a  -- | 'lookAheadE' runs the @todo@ action. If the action returns 'Left' then the @@ -176,7 +176,7 @@ lookAheadE todo = do   setCheckpoint   a <- todo-  either (\_ -> useCheckpoint) (\_ -> clearCheckpoint) a+  either (const useCheckpoint) (const clearCheckpoint) a   return a  -- 'collect' is used by 'putCheckpoint' and 'throwError'@@ -249,8 +249,7 @@ suspendUntilComplete :: Get () suspendUntilComplete = do   continue <- suspend-  if continue then suspendUntilComplete-    else return ()+  when continue suspendUntilComplete  -- | Call suspend and throw and error with the provided @msg@ if -- Nothing has been passed to the 'Partial' continuation.  Otherwise@@ -541,7 +540,7 @@   {-# INLINE return #-}   m >>= k  = Get (\sc -> unGet m (\a -> unGet (k a) sc))   {-# INLINE (>>=) #-}-  fail msg = throwError (strMsg msg)+  fail = throwError . strMsg  instance MonadError String Get where   throwError msg = Get $ \_sc  s pcIn ->
Text/ProtocolBuffers/Identifiers.hs view
@@ -74,7 +74,7 @@  {-# INLINE readIt #-} readIt :: (Read a) => (a -> a1) -> String -> Int -> String -> [(a1, String)]-readIt con name d  = readParen (d > app_prec) (\r -> [(con m,t) | (name',s) <- lex r, name==name', (m,t) <- readsPrec (max_prec) s])+readIt con name d  = readParen (d > app_prec) (\r -> [(con m,t) | (name',s) <- lex r, name==name', (m,t) <- readsPrec max_prec s])  {-# INLINE showIt #-} showIt :: (Show a) => Int -> [Char] -> a -> String -> String
Text/ProtocolBuffers/WireMessage.hs view
@@ -36,14 +36,15 @@     ) where  import Control.Monad(when)-import Control.Monad.Error.Class(throwError,catchError)+import Control.Monad.Error.Class(throwError) import Control.Monad.ST import Data.Array.ST import Data.Bits (Bits(..)) import qualified Data.ByteString.Lazy as BS (length) import qualified Data.Foldable as F(foldl',forM_) import Data.List (genericLength)-import qualified Data.Set as Set(delete,null,notMember)+import Data.Maybe(fromMaybe)+import qualified Data.Set as Set(delete,null) import Data.Typeable (Typeable(..)) -- GHC internals for getting at Double and Float representation as Word64 and Word32 -- This has been superceded by the ST array trick (ugly, but promised to work)@@ -123,7 +124,7 @@ -- -- This is 'runGetOnLazy' applied to 'messageGetM'. messageGet :: (ReflectDescriptor msg, Wire msg) => ByteString -> Either String (msg,ByteString)-messageGet bs = runGetOnLazy (messageGetM) bs+messageGet bs = runGetOnLazy messageGetM bs  -- | This 'runGetOnLazy' applied to 'messageWithLengthGetM'. --@@ -131,7 +132,7 @@ -- succeed when it has consumed precisely this many additional bytes. -- The 'ByteString' after this point will be returned. messageWithLengthGet :: (ReflectDescriptor msg, Wire msg) => ByteString -> Either String (msg,ByteString)-messageWithLengthGet bs = runGetOnLazy (messageWithLengthGetM) bs+messageWithLengthGet bs = runGetOnLazy messageWithLengthGetM bs  -- | This reads the tagged message fields until the stop tag or the -- end of input is reached.@@ -486,7 +487,7 @@     Nothing -> throwError (msg ++ show int)  where msg = "Bad wireGet of Enum "++show (typeOf (undefined `asTypeOf` typeHack toMaybe'Enum))++", unrecognized Int value is "        typeHack :: (Int -> Maybe e) -> e-       typeHack f = maybe undefined id (f undefined)+       typeHack f = fromMaybe undefined (f undefined)  -- This will have to examine the value of positive numbers to get the size {-# INLINE size'Varint #-}
− descriptor.proto 
@@ -1,286 +0,0 @@-// Protocol Buffers - Google's data interchange format-// Copyright 2008 Google Inc.-// http://code.google.com/p/protobuf/-//-// Licensed under the Apache License, Version 2.0 (the "License");-// you may not use this file except in compliance with the License.-// You may obtain a copy of the License at-//-//      http://www.apache.org/licenses/LICENSE-2.0-//-// Unless required by applicable law or agreed to in writing, software-// distributed under the License is distributed on an "AS IS" BASIS,-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.-// See the License for the specific language governing permissions and-// limitations under the License.--// Author: kenton@google.com (Kenton Varda)-//  Based on original Protocol Buffers design by-//  Sanjay Ghemawat, Jeff Dean, and others.-//-// The messages in this file describe the definitions found in .proto files.-// A valid .proto file can be translated directly to a FileDescriptorProto-// without any other information (e.g. without reading its imports).----package google.protobuf;-option java_package = "com.google.protobuf";-option java_outer_classname = "DescriptorProtos";--// descriptor.proto must be optimized for speed because reflection-based-// algorithms don't work during bootstrapping.-option optimize_for = SPEED;--// Describes a complete .proto file.-message FileDescriptorProto {-  optional string name = 1;       // file name, relative to root of source tree-  optional string package = 2;    // e.g. "foo", "foo.bar", etc.--  // Names of files imported by this file.-  repeated string dependency = 3;--  // All top-level definitions in this file.-  repeated DescriptorProto message_type = 4;-  repeated EnumDescriptorProto enum_type = 5;-  repeated ServiceDescriptorProto service = 6;-  repeated FieldDescriptorProto extension = 7;--  optional FileOptions options = 8;-}--// Describes a message type.-message DescriptorProto {-  optional string name = 1;--  repeated FieldDescriptorProto field = 2;-  repeated FieldDescriptorProto extension = 6;--  repeated DescriptorProto nested_type = 3;-  repeated EnumDescriptorProto enum_type = 4;--  message ExtensionRange {-    optional int32 start = 1;-    optional int32 end = 2;-  }-  repeated ExtensionRange extension_range = 5;--  optional MessageOptions options = 7;-}--// Describes a field within a message.-message FieldDescriptorProto {-  enum Type {-    // 0 is reserved for errors.-    // Order is weird for historical reasons.-    TYPE_DOUBLE         = 1;-    TYPE_FLOAT          = 2;-    TYPE_INT64          = 3;   // Not ZigZag encoded.  Negative numbers-                               // take 10 bytes.  Use TYPE_SINT64 if negative-                               // values are likely.-    TYPE_UINT64         = 4;-    TYPE_INT32          = 5;   // Not ZigZag encoded.  Negative numbers-                               // take 10 bytes.  Use TYPE_SINT32 if negative-                               // values are likely.-    TYPE_FIXED64        = 6;-    TYPE_FIXED32        = 7;-    TYPE_BOOL           = 8;-    TYPE_STRING         = 9;-    TYPE_GROUP          = 10;  // Tag-delimited aggregate.-    TYPE_MESSAGE        = 11;  // Length-delimited aggregate.--    // New in version 2.-    TYPE_BYTES          = 12;-    TYPE_UINT32         = 13;-    TYPE_ENUM           = 14;-    TYPE_SFIXED32       = 15;-    TYPE_SFIXED64       = 16;-    TYPE_SINT32         = 17;  // Uses ZigZag encoding.-    TYPE_SINT64         = 18;  // Uses ZigZag encoding.-  };--  enum Label {-    // 0 is reserved for errors-    LABEL_OPTIONAL      = 1;-    LABEL_REQUIRED      = 2;-    LABEL_REPEATED      = 3;-    // TODO(sanjay): Should we add LABEL_MAP?-  };--  optional string name = 1;-  optional int32 number = 3;-  optional Label label = 4;--  // If type_name is set, this need not be set.  If both this and type_name-  // are set, this must be either TYPE_ENUM or TYPE_MESSAGE.-  optional Type type = 5;--  // For message and enum types, this is the name of the type.  If the name-  // starts with a '.', it is fully-qualified.  Otherwise, C++-like scoping-  // rules are used to find the type (i.e. first the nested types within this-  // message are searched, then within the parent, on up to the root-  // namespace).-  optional string type_name = 6;--  // For extensions, this is the name of the type being extended.  It is-  // resolved in the same manner as type_name.-  optional string extendee = 2;--  // For numeric types, contains the original text representation of the value.-  // For booleans, "true" or "false".-  // For strings, contains the default text contents (not escaped in any way).-  // For bytes, contains the C escaped value.  All bytes >= 128 are escaped.-  // TODO(kenton):  Base-64 encode?-  optional string default_value = 7;--  optional FieldOptions options = 8;-}--// Describes an enum type.-message EnumDescriptorProto {-  optional string name = 1;--  repeated EnumValueDescriptorProto value = 2;--  optional EnumOptions options = 3;-}--// Describes a value within an enum.-message EnumValueDescriptorProto {-  optional string name = 1;-  optional int32 number = 2;--  optional EnumValueOptions options = 3;-}--// Describes a service.-message ServiceDescriptorProto {-  optional string name = 1;-  repeated MethodDescriptorProto method = 2;--  optional ServiceOptions options = 3;-}--// Describes a method of a service.-message MethodDescriptorProto {-  optional string name = 1;--  // Input and output type names.  These are resolved in the same way as-  // FieldDescriptorProto.type_name, but must refer to a message type.-  optional string input_type = 2;-  optional string output_type = 3;--  optional MethodOptions options = 4;-}--// ===================================================================-// Options--// Each of the definitions above may have "options" attached.  These are-// just annotations which may cause code to be generated slightly differently-// or may contain hints for code that manipulates protocol messages.--// TODO(kenton):  Allow extensions to options.--message FileOptions {--  // Sets the Java package where classes generated from this .proto will be-  // placed.  By default, the proto package is used, but this is often-  // inappropriate because proto packages do not normally start with backwards-  // domain names.-  optional string java_package = 1;---  // If set, all the classes from the .proto file are wrapped in a single-  // outer class with the given name.  This applies to both Proto1-  // (equivalent to the old "--one_java_file" option) and Proto2 (where-  // a .proto always translates to a single class, but you may want to-  // explicitly choose the class name).-  optional string java_outer_classname = 8;--  // If set true, then the Java code generator will generate a separate .java-  // file for each top-level message, enum, and service defined in the .proto-  // file.  Thus, these types will *not* be nested inside the outer class-  // named by java_outer_classname.  However, the outer class will still be-  // generated to contain the file's getDescriptor() method as well as any-  // top-level extensions defined in the file.-  optional bool java_multiple_files = 10 [default=false];--  // Generated classes can be optimized for speed or code size.-  enum OptimizeMode {-    SPEED = 1;      // Generate complete code for parsing, serialization, etc.-    CODE_SIZE = 2;  // Use ReflectionOps to implement these methods.-  }-  optional OptimizeMode optimize_for = 9 [default=CODE_SIZE];-}--message MessageOptions {-  // Set true to use the old proto1 MessageSet wire format for extensions.-  // This is provided for backwards-compatibility with the MessageSet wire-  // format.  You should not use this for any other reason:  It's less-  // efficient, has fewer features, and is more complicated.-  //-  // The message must be defined exactly as follows:-  //   message Foo {-  //     option message_set_wire_format = true;-  //     extensions 4 to max;-  //   }-  // Note that the message cannot have any defined fields; MessageSets only-  // have extensions.-  //-  // All extensions of your type must be singular messages; e.g. they cannot-  // be int32s, enums, or repeated messages.-  //-  // Because this is an option, the above two restrictions are not enforced by-  // the protocol compiler.-  optional bool message_set_wire_format = 1 [default=false];-}--message FieldOptions {-  // The ctype option instructs the C++ code generator to use a different-  // representation of the field than it normally would.  See the specific-  // options below.  This option is not yet implemented in the open source-  // release -- sorry, we'll try to include it in a future version!-  optional CType ctype = 1;-  enum CType {-    CORD = 1;--    STRING_PIECE = 2;-  }--  // EXPERIMENTAL.  DO NOT USE.-  // For "map" fields, the name of the field in the enclosed type that-  // is the key for this map.  For example, suppose we have:-  //   message Item {-  //     required string name = 1;-  //     required string value = 2;-  //   }-  //   message Config {-  //     repeated Item items = 1 [experimental_map_key="name"];-  //   }-  // In this situation, the map key for Item will be set to "name".-  // TODO: Fully-implement this, then remove the "experimental_" prefix.-  optional string experimental_map_key = 9;-}--message EnumOptions {-}--message EnumValueOptions {-}--message ServiceOptions {--  // Note:  Field numbers 1 through 32 are reserved for Google's internal RPC-  //   framework.  We apologize for hoarding these numbers to ourselves, but-  //   we were already using them long before we decided to release Protocol-  //   Buffers.-}--message MethodOptions {--  // Note:  Field numbers 1 through 32 are reserved for Google's internal RPC-  //   framework.  We apologize for hoarding these numbers to ourselves, but-  //   we were already using them long before we decided to release Protocol-  //   Buffers.-}
− doc.txt 
@@ -1,120 +0,0 @@-By Chris Kukelwicz, started 2008-07-08------ In Progress--Make Gen.hs generate trivial Get.hs instances for EnumDescriptorProto and DescriptorProtos.-  (will be simply get f m = f m, or get = ($))--I think Gen.hs has all I need for bootstrapping.-Now needed is-  (1) A .proto file parser or lexer/parser that works on descriptor.proto (take old Parsec or new Alex?)-  (2) A name-resolution pass to turn all message & enum references into fully qualified names.-  (3) A mangling pass to make all names valid in haskell-  (4) Ensure the needed fields are set (name/number/label/type, and when relevant, the type_name)--From there the Gen.hs can generate the module text, only an output routine is missing.--At that point the basic system will be bootstrapped.  --DONE-(*) Create the wire format and generate Wire instances--Then TODO is still quite long-(*) Test Wire instances-(*) continuation based Get-(*) save decomposed reals instead of rationals-(*) Add more support to .proto loading-   (**) extension support-   (**) groups-   (**) service/rpc-(*) Add support to Gen.hs code generation-   (**) Get.hs style API instances-   (**) extension, group, service support-   (**) detect and handle mutually recursive module references-(*) RPC API and instances---------wire format: Using Data.Binary.Get/Put/Builder will make this trivial.-  Only ugly piece is Double <-> [Word8] which I have prototyped in WireMessage.hs--------I am uploading this to hackage early, in case someone wished to avoid-duplicating effort.--I am loeading version 0.0.2 with--cd src-ghci -XTemplateHaskell -XEmptyDataDecls -XGADTs -XFlexibleInstances -XDeriveDataTypeable ProtocolBuffers/ParseProto.hs--Which ensure that DescriptorProtos/* all compile and tests the stub-like parsec parser (runs agains the "test" file).--http://code.google.com/apis/protocolbuffers/docs/overview.html-http://code.google.com/p/protobuf/downloads/list-http://groups.google.com/group/protobuf-http://code.google.com/apis/protocolbuffers/docs/proto.html--In particular you need 'descriptor.proto' from the source from Google.-This file describes the programatic representation of a ".proto" file.--to bootstrap:--DONE (1) Decide on how to make the namespaces work-DONE (2) Manually translate 'descriptor.proto' into basic data types under its "DescriptorProtos" namespace-     (3) Write Parsec parser that can load 'descriptor.proto' into basic data types from (2)-     (4) Write automatic translator that can generate haskell source to replace (2) using parsed data from (3)-     (5) Expand (3) to handle full specification for '.proto' files-     (5) Add API (see Java/Python/C++ APIs) to manipulate data/messages/enums/groups/extensions-     (6) Expand (4) to create stub instances for API in (5)-     (7) Implement stub instances for serializing to and from the wire (use Data.Binary ?)-     (8) Implement stub instances for 'rpc' calls--The self-hosting nature of "DescriptorProtos" means that opportunities-for type safety are being lost.  The ranges of the integers and-contents of the strings are not part of the '.proto' file, so the-generated data structures do not reflect bounds or encodings.  The-parser in (3) can be tweaked to use more fine-grained (new)types.--MONOID !--The text at http://code.google.com/apis/protocolbuffers/docs/encoding.html#optional specifies that Messages are Monoids!-And the rule is simple: Take the last value in the case of conflicts, or concatenate if repeated.--Rather than use-"type MyMaybe a = (Data.Monoid.Last (Data.Maybe.Maybe a))"-I have create ProtocolBuffers.Option which uses a phantom type to record whether it is required,-and it has the last-biased mappend semantics.--Design issues for generated Haskell code:-- * Wire protocol reading-   http://code.google.com/apis/protocolbuffers/docs/encoding.html--  The wire protocol decodes a fixed length buffer into a Message.--  The wire protocol reader can break the input into a top level series of (field#,wireType#,data#)-    field# is the 29 bit: 0 to 2^29-1 field #-    wireType# is 3 bit value (currently Varint, 64-Bit, string, endian 32-Bit)-    data# is decoded to some (Varint, Word64, ByteString, Word32)--  see src/ProtocolBuffers/WireMessage.hs for an intermediate repesentation in Haskell-- * Generating Haskell data types--  The parsing result may have unknown fields which will need to be stored somewhere.-  These come of the Wire, and can only be stored as a left over ProtocolBuffers.WireMessage--  Should the generated class use "Maybe" for optional fields?-  If Yes then-    => This implies a new type class "Mergable" which is isomorphic to Data.Monoid but with instances needed to merge messages.--  Should the generated class use "Maybe" for required fields?-  If No then-    => This implies the API cannot return a partially filled in message-      => This implies you cannot split the encoded bytes into two pieces then decode them into two messages and finially merge the two message, because one of the halves will be missing a required field!-  If Yes then-    => This implies that the type are more complicated and invalid message can be repesented.--"If an invalid enum value is read when parsing a message, it will be treated as an unknown field."
+ google/protobuf/unittest_custom_options.proto view
@@ -0,0 +1,271 @@+// Protocol Buffers - Google's data interchange format+// Copyright 2008 Google Inc.  All rights reserved.+// http://code.google.com/p/protobuf/+//+// Redistribution and use in source and binary forms, with or without+// modification, are permitted provided that the following conditions are+// met:+//+//     * Redistributions of source code must retain the above copyright+// notice, this list of conditions and the following disclaimer.+//     * Redistributions in binary form must reproduce the above+// copyright notice, this list of conditions and the following disclaimer+// in the documentation and/or other materials provided with the+// distribution.+//     * Neither the name of Google Inc. nor the names of its+// contributors may be used to endorse or promote products derived from+// this software without specific prior written permission.+//+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS+// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT+// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR+// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT+// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT+// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,+// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY+// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.++// Author: benjy@google.com (Benjy Weinberger)+//  Based on original Protocol Buffers design by+//  Sanjay Ghemawat, Jeff Dean, and others.+//+// A proto file used to test the "custom options" feature of proto2.+++// A custom file option (defined below).+option (file_opt1) = 9876543210;++import "google/protobuf/descriptor.proto";++// We don't put this in a package within proto2 because we need to make sure+// that the generated code doesn't depend on being in the proto2 namespace.+package protobuf_unittest;+++// Some simple test custom options of various types.++extend google.protobuf.FileOptions {+  optional uint64 file_opt1 = 7736974;+}++extend google.protobuf.MessageOptions {+  optional int32 message_opt1 = 7739036;+}++extend google.protobuf.FieldOptions {+  optional fixed64 field_opt1 = 7740936;+  // This is useful for testing that we correctly register default values for+  // extension options.+  optional int32 field_opt2 = 7753913 [default=42];+}++extend google.protobuf.EnumOptions {+  optional sfixed32 enum_opt1 = 7753576;+}++// TODO(benjy): Test options on enum values when the parser supports them.++extend google.protobuf.ServiceOptions {+  optional sint64 service_opt1 = 7887650;+}++enum MethodOpt1 {+  METHODOPT1_VAL1 = 1;+  METHODOPT1_VAL2 = 2;+}++extend google.protobuf.MethodOptions {+  optional MethodOpt1 method_opt1 = 7890860;+}++// A test message with custom options at all possible locations (and also some+// regular options, to make sure they interact nicely).+message TestMessageWithCustomOptions {+  option message_set_wire_format = false;++  option (message_opt1) = -56;++  optional string field1 = 1 [ctype=CORD,+                              (field_opt1)=8765432109];++  enum AnEnum {+    option (enum_opt1) = -789;++    ANENUM_VAL1 = 1;+    ANENUM_VAL2 = 2;+  }+}+++// A test RPC service with custom options at all possible locations (and also+// some regular options, to make sure they interact nicely).+message CustomOptionFooRequest {+}++message CustomOptionFooResponse {+}++service TestServiceWithCustomOptions {+  option (service_opt1) = -9876543210;++  rpc Foo(CustomOptionFooRequest) returns (CustomOptionFooResponse) {+    option (method_opt1) = METHODOPT1_VAL2;+  }+}++++// Options of every possible field type, so we can test them all exhaustively.++message DummyMessageContainingEnum {+  enum TestEnumType {+    TEST_OPTION_ENUM_TYPE1 = 22;+    TEST_OPTION_ENUM_TYPE2 = -23;+  }+}++message DummyMessageInvalidAsOptionType {+}++extend google.protobuf.MessageOptions {+  optional         bool     bool_opt = 7706090;+  optional        int32    int32_opt = 7705709;+  optional        int64    int64_opt = 7705542;+  optional       uint32   uint32_opt = 7704880;+  optional       uint64   uint64_opt = 7702367;+  optional       sint32   sint32_opt = 7701568;+  optional       sint64   sint64_opt = 7700863;+  optional      fixed32  fixed32_opt = 7700307;+  optional      fixed64  fixed64_opt = 7700194;+  optional     sfixed32 sfixed32_opt = 7698645;+  optional     sfixed64 sfixed64_opt = 7685475;+  optional        float    float_opt = 7675390;+  optional       double   double_opt = 7673293;+  optional       string   string_opt = 7673285;+  optional        bytes    bytes_opt = 7673238;+  optional DummyMessageContainingEnum.TestEnumType enum_opt = 7673233;+  optional DummyMessageInvalidAsOptionType message_type_opt = 7665967;+}++message CustomOptionMinIntegerValues {+  option     (bool_opt) = false;+  option    (int32_opt) = -0x80000000;+  option    (int64_opt) = -0x8000000000000000;+  option   (uint32_opt) = 0;+  option   (uint64_opt) = 0;+  option   (sint32_opt) = -0x80000000;+  option   (sint64_opt) = -0x8000000000000000;+  option  (fixed32_opt) = 0;+  option  (fixed64_opt) = 0;+  option (sfixed32_opt) = -0x80000000;+  option (sfixed64_opt) = -0x8000000000000000;+}++message CustomOptionMaxIntegerValues {+  option     (bool_opt) = true;+  option    (int32_opt) = 0x7FFFFFFF;+  option    (int64_opt) = 0x7FFFFFFFFFFFFFFF;+  option   (uint32_opt) = 0xFFFFFFFF;+  option   (uint64_opt) = 0xFFFFFFFFFFFFFFFF;+  option   (sint32_opt) = 0x7FFFFFFF;+  option   (sint64_opt) = 0x7FFFFFFFFFFFFFFF;+  option  (fixed32_opt) = 0xFFFFFFFF;+  option  (fixed64_opt) = 0xFFFFFFFFFFFFFFFF;+  option (sfixed32_opt) = 0x7FFFFFFF;+  option (sfixed64_opt) = 0x7FFFFFFFFFFFFFFF;+}++message CustomOptionOtherValues {+  option  (int32_opt) = -100;  // To test sign-extension.+  option  (float_opt) = 12.3456789;+  option (double_opt) = 1.234567890123456789;+  option (string_opt) = "Hello, \"World\"";+  option  (bytes_opt) = "Hello\0World";+  option   (enum_opt) = TEST_OPTION_ENUM_TYPE2;+}++message SettingRealsFromPositiveInts {+  option  (float_opt) = 12;+  option (double_opt) = 154;+}++message SettingRealsFromNegativeInts {+  option  (float_opt) = -12;+  option  (double_opt) = -154;+}++// Options of complex message types, themselves combined and extended in+// various ways.++message ComplexOptionType1 {+  optional int32 foo = 1;++  extensions 100 to max;+}++message ComplexOptionType2 {+  optional ComplexOptionType1 bar = 1;+  optional int32 baz = 2;++  message ComplexOptionType4 {+    optional int32 waldo = 1;++    extend google.protobuf.MessageOptions {+      optional ComplexOptionType4 complex_opt4 = 7633546;+    }+  }++  optional ComplexOptionType4 fred = 3;++  extensions 100 to max;+}++message ComplexOptionType3 {+  optional int32 qux = 1;++  optional group ComplexOptionType5 = 2 {+    optional int32 plugh = 3;+  }+}++extend ComplexOptionType1 {+  optional int32 quux = 7663707;+  optional ComplexOptionType3 corge = 7663442;+}++extend ComplexOptionType2 {+  optional int32 grault = 7650927;+  optional ComplexOptionType1 garply = 7649992;+}++extend google.protobuf.MessageOptions {+  optional protobuf_unittest.ComplexOptionType1 complex_opt1 = 7646756;+  optional ComplexOptionType2 complex_opt2 = 7636949;+  optional ComplexOptionType3 complex_opt3 = 7636463;+  optional group ComplexOpt6 = 7595468 {+    optional int32 xyzzy = 7593951;+  }+}++// Note that we try various different ways of naming the same extension.+message VariousComplexOptions {+  option (.protobuf_unittest.complex_opt1).foo = 42;+  option (protobuf_unittest.complex_opt1).(.protobuf_unittest.quux) = 324;+  option (.protobuf_unittest.complex_opt1).(protobuf_unittest.corge).qux = 876;+  option (complex_opt2).baz = 987;+  option (complex_opt2).(grault) = 654;+  option (complex_opt2).bar.foo = 743;+  option (complex_opt2).bar.(quux) = 1999;+  option (complex_opt2).bar.(protobuf_unittest.corge).qux = 2008;+  option (complex_opt2).(garply).foo = 741;+  option (complex_opt2).(garply).(.protobuf_unittest.quux) = 1998;+  option (complex_opt2).(protobuf_unittest.garply).(corge).qux = 2121;+  option (ComplexOptionType2.ComplexOptionType4.complex_opt4).waldo = 1971;+  option (complex_opt2).fred.waldo = 321;+  option (protobuf_unittest.complex_opt3).qux = 9;+  option (complex_opt3).complexoptiontype5.plugh = 22;+  option (complexopt6).xyzzy = 24;+}
protocol-buffers.cabal view
@@ -1,28 +1,26 @@ name:           protocol-buffers-version:        1.2.2-cabal-version:  >= 1.2+version:        1.4.0+cabal-version:  >= 1.6 build-type:     Simple license:        BSD3 license-file:   LICENSE copyright:      (c) 2008 Christopher Edward Kuklewicz author:         Christopher Edward Kuklewicz maintainer:     Chris Kuklewicz <protobuf@personal.mightyreason.com>-stability:      Experimental+stability:      Beta homepage:       http://hackage.haskell.org/cgi-bin/hackage-scripts/package/protocol-buffers package-url:    http://darcs.haskell.org/packages/protocol-buffers2/ synopsis:       Parse Google Protocol Buffer specifications description:    Parse proto files and generate Haskell code. category:       Text-Tested-With:    GHC ==6.8.3+Tested-With:    GHC ==6.10.1 --data-files:      extra-source-files: Setup.hs-                    doc.txt                     TODO                     README-                    Skeleton.hs-boot-                    descriptor.proto                     google/protobuf/unittest.proto                     google/protobuf/unittest_import.proto+                    google/protobuf/unittest_custom_options.proto -- extra-tmp-files:  flag small_base@@ -42,27 +40,19 @@                    Text.ProtocolBuffers.WireMessage                    Text.ProtocolBuffers.Identifiers +  build-depends: containers,+                 bytestring,+                 array,+                 filepath,+                 directory,+                 mtl,+                 binary, parsec, utf8-string, haskell-src,+                 QuickCheck+   if flag(small_base)-    build-depends: base == 4.*,-                   containers,-                   bytestring,-                   array,-                   filepath,-                   directory,-                   mtl,-                   QuickCheck,-                   binary, parsec, utf8-string, haskell-src,-                   syb+    build-depends: base == 4.*, syb   else-    build-depends: base == 3.*,-                   containers,-                   bytestring,-                   array,-                   filepath,-                   directory,-                   mtl,-                   binary, parsec, utf8-string, haskell-src,-                   QuickCheck+    build-depends: base == 3.*  -- Most of these are needed for protocol-buffers (Get and WireMessage.hs)   extensions:      DeriveDataTypeable,