diff --git a/README.md b/README.md --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ ## Version -`1.3.1` +`1.3.2` ## Description diff --git a/amazonka-kinesis.cabal b/amazonka-kinesis.cabal --- a/amazonka-kinesis.cabal +++ b/amazonka-kinesis.cabal @@ -1,5 +1,5 @@ name: amazonka-kinesis -version: 1.3.1 +version: 1.3.2 synopsis: Amazon Kinesis SDK. homepage: https://github.com/brendanhay/amazonka bug-reports: https://github.com/brendanhay/amazonka/issues @@ -65,7 +65,7 @@ , Network.AWS.Kinesis.Types.Sum build-depends: - amazonka-core == 1.3.1.* + amazonka-core == 1.3.2.* , base >= 4.7 && < 5 test-suite amazonka-kinesis-test @@ -85,9 +85,9 @@ , Test.AWS.Kinesis.Internal build-depends: - amazonka-core == 1.3.1.* - , amazonka-test == 1.3.1.* - , amazonka-kinesis == 1.3.1.* + amazonka-core == 1.3.2.* + , amazonka-test == 1.3.2.* + , amazonka-kinesis == 1.3.2.* , base , bytestring , lens diff --git a/gen/Network/AWS/Kinesis.hs b/gen/Network/AWS/Kinesis.hs --- a/gen/Network/AWS/Kinesis.hs +++ b/gen/Network/AWS/Kinesis.hs @@ -123,6 +123,7 @@ -- ** Record , Record , record + , rApproximateArrivalTimestamp , rSequenceNumber , rData , rPartitionKey diff --git a/gen/Network/AWS/Kinesis/CreateStream.hs b/gen/Network/AWS/Kinesis/CreateStream.hs --- a/gen/Network/AWS/Kinesis/CreateStream.hs +++ b/gen/Network/AWS/Kinesis/CreateStream.hs @@ -25,12 +25,12 @@ -- data records in an Amazon Kinesis stream. -- -- You specify and control the number of shards that a stream is composed --- of. Each open shard can support up to 5 read transactions per second, up --- to a maximum total of 2 MB of data read per second. Each shard can --- support up to 1000 records written per second, up to a maximum total of --- 1 MB data written per second. You can add shards to a stream if the --- amount of data input increases and you can remove shards if the amount --- of data input decreases. +-- of. Each shard can support reads up to 5 transactions per second, up to +-- a maximum data read total of 2 MB per second. Each shard can support +-- writes up to 1,000 records per second, up to a maximum data write total +-- of 1 MB per second. You can add shards to a stream if the amount of data +-- input increases and you can remove shards if the amount of data input +-- decreases. -- -- The stream name identifies the stream. The name is scoped to the AWS -- account used by the application. It is also scoped by region. That is, @@ -54,7 +54,7 @@ -- For the default shard limit for an AWS account, see -- . -- If you need to increase this limit, --- +-- . -- -- You can use 'DescribeStream' to check the stream status, which is -- returned in 'StreamStatus'. diff --git a/gen/Network/AWS/Kinesis/GetRecords.hs b/gen/Network/AWS/Kinesis/GetRecords.hs --- a/gen/Network/AWS/Kinesis/GetRecords.hs +++ b/gen/Network/AWS/Kinesis/GetRecords.hs @@ -40,13 +40,11 @@ -- record with the sequence number or other attribute that marks it as the -- last record to process. -- --- Each data record can be up to 50 KB in size, and each shard can read up +-- Each data record can be up to 1 MB in size, and each shard can read up -- to 2 MB per second. You can ensure that your calls don\'t exceed the -- maximum supported size or throughput by using the 'Limit' parameter to -- specify the maximum number of records that GetRecords can return. --- Consider your average record size when determining this limit. For --- example, if your average record size is 40 KB, you can limit the data --- returned to about 1 MB per call by specifying 25 as the limit. +-- Consider your average record size when determining this limit. -- -- The size of the data returned by GetRecords will vary depending on the -- utilization of the shard. The maximum size of data that GetRecords can @@ -62,10 +60,19 @@ -- -- To detect whether the application is falling behind in processing, you -- can use the 'MillisBehindLatest' response attribute. You can also --- monitor the amount of data in a stream using the CloudWatch metrics. For --- more information, see --- --- in the /Amazon Kinesis Developer Guide/. +-- monitor the stream using CloudWatch metrics (see +-- +-- in the /Amazon Kinesis Developer Guide/). +-- +-- Each Amazon Kinesis record includes a value, +-- 'ApproximateArrivalTimestamp', that is set when an Amazon Kinesis stream +-- successfully receives and stores a record. This is commonly referred to +-- as a server-side timestamp, which is different than a client-side +-- timestamp, where the timestamp is set when a data producer creates or +-- sends the record to a stream. The timestamp has millisecond precision. +-- There are no guarantees about the timestamp accuracy, or that the +-- timestamp is always increasing. For example, records in a shard or +-- across a stream might have timestamps that are out of order. -- -- /See:/ for GetRecords. module Network.AWS.Kinesis.GetRecords diff --git a/gen/Network/AWS/Kinesis/PutRecord.hs b/gen/Network/AWS/Kinesis/PutRecord.hs --- a/gen/Network/AWS/Kinesis/PutRecord.hs +++ b/gen/Network/AWS/Kinesis/PutRecord.hs @@ -18,11 +18,11 @@ -- Stability : auto-generated -- Portability : non-portable (GHC extensions) -- --- Puts (writes) a single data record from a producer into an Amazon --- Kinesis stream. Call 'PutRecord' to send data from the producer into the --- Amazon Kinesis stream for real-time ingestion and subsequent processing, --- one record at a time. Each shard can support up to 1000 records written --- per second, up to a maximum total of 1 MB data written per second. +-- Writes a single data record from a producer into an Amazon Kinesis +-- stream. Call 'PutRecord' to send data from the producer into the Amazon +-- Kinesis stream for real-time ingestion and subsequent processing, one +-- record at a time. Each shard can support writes up to 1,000 records per +-- second, up to a maximum data write total of 1 MB per second. -- -- You must specify the name of the stream that captures, stores, and -- transports the data; a partition key; and the data blob itself. @@ -42,7 +42,7 @@ -- shards using the hash key ranges of the shards. You can override hashing -- the partition key to determine the shard by explicitly specifying a hash -- value using the 'ExplicitHashKey' parameter. For more information, see --- +-- -- in the /Amazon Kinesis Developer Guide/. -- -- 'PutRecord' returns the shard ID of where the data record was placed and @@ -51,7 +51,7 @@ -- Sequence numbers generally increase over time. To guarantee strictly -- increasing ordering, use the 'SequenceNumberForOrdering' parameter. For -- more information, see --- +-- -- in the /Amazon Kinesis Developer Guide/. -- -- If a 'PutRecord' request cannot be processed because of insufficient @@ -146,8 +146,9 @@ prStreamName = lens _prStreamName (\ s a -> s{_prStreamName = a}); -- | The data blob to put into the record, which is base64-encoded when the --- blob is serialized. The maximum size of the data blob (the payload --- before base64-encoding) is 50 kilobytes (KB) +-- blob is serialized. When the data blob (the payload before +-- base64-encoding) is added to the partition key size, the total size must +-- not exceed the maximum record size (1 MB). -- -- /Note:/ This 'Lens' automatically encodes and decodes Base64 data, -- despite what the AWS documentation might say. diff --git a/gen/Network/AWS/Kinesis/PutRecords.hs b/gen/Network/AWS/Kinesis/PutRecords.hs --- a/gen/Network/AWS/Kinesis/PutRecords.hs +++ b/gen/Network/AWS/Kinesis/PutRecords.hs @@ -18,16 +18,21 @@ -- Stability : auto-generated -- Portability : non-portable (GHC extensions) -- --- Puts (writes) multiple data records from a producer into an Amazon --- Kinesis stream in a single call (also referred to as a 'PutRecords' --- request). Use this operation to send data from a data producer into the --- Amazon Kinesis stream for real-time ingestion and processing. Each shard --- can support up to 1000 records written per second, up to a maximum total --- of 1 MB data written per second. +-- Writes multiple data records from a producer into an Amazon Kinesis +-- stream in a single call (also referred to as a 'PutRecords' request). +-- Use this operation to send data from a data producer into the Amazon +-- Kinesis stream for data ingestion and processing. -- +-- Each 'PutRecords' request can support up to 500 records. Each record in +-- the request can be as large as 1 MB, up to a limit of 5 MB for the +-- entire request, including partition keys. Each shard can support writes +-- up to 1,000 records per second, up to a maximum data write total of 1 MB +-- per second. +-- -- You must specify the name of the stream that captures, stores, and -- transports the data; and an array of request 'Records', with each record --- in the array requiring a partition key and data blob. +-- in the array requiring a partition key and data blob. The record size +-- limit applies to the total size of the partition key and data blob. -- -- The data blob can be any type of data; for example, a segment from a log -- file, geographic\/location data, website clickstream data, and so on. @@ -38,14 +43,14 @@ -- values and to map associated data records to shards. As a result of this -- hashing mechanism, all data records with the same partition key map to -- the same shard within the stream. For more information, see --- +-- -- in the /Amazon Kinesis Developer Guide/. -- -- Each record in the 'Records' array may include an optional parameter, -- 'ExplicitHashKey', which overrides the partition key to shard mapping. -- This parameter allows a data producer to determine explicitly the shard -- where the record is stored. For more information, see --- +-- -- in the /Amazon Kinesis Developer Guide/. -- -- The 'PutRecords' response includes an array of response 'Records'. Each diff --git a/gen/Network/AWS/Kinesis/SplitShard.hs b/gen/Network/AWS/Kinesis/SplitShard.hs --- a/gen/Network/AWS/Kinesis/SplitShard.hs +++ b/gen/Network/AWS/Kinesis/SplitShard.hs @@ -60,7 +60,7 @@ -- For the default shard limit for an AWS account, see -- . -- If you need to increase this limit, --- +-- . -- -- If you try to operate on too many streams in parallel using -- CreateStream, DeleteStream, MergeShards or SplitShard, you receive a diff --git a/gen/Network/AWS/Kinesis/Types.hs b/gen/Network/AWS/Kinesis/Types.hs --- a/gen/Network/AWS/Kinesis/Types.hs +++ b/gen/Network/AWS/Kinesis/Types.hs @@ -53,6 +53,7 @@ -- * Record , Record , record + , rApproximateArrivalTimestamp , rSequenceNumber , rData , rPartitionKey diff --git a/gen/Network/AWS/Kinesis/Types/Product.hs b/gen/Network/AWS/Kinesis/Types/Product.hs --- a/gen/Network/AWS/Kinesis/Types/Product.hs +++ b/gen/Network/AWS/Kinesis/Types/Product.hs @@ -96,8 +96,9 @@ prreExplicitHashKey = lens _prreExplicitHashKey (\ s a -> s{_prreExplicitHashKey = a}); -- | The data blob to put into the record, which is base64-encoded when the --- blob is serialized. The maximum size of the data blob (the payload --- before base64-encoding) is 50 kilobytes (KB) +-- blob is serialized. When the data blob (the payload before +-- base64-encoding) is added to the partition key size, the total size must +-- not exceed the maximum record size (1 MB). -- -- /Note:/ This 'Lens' automatically encodes and decodes Base64 data, -- despite what the AWS documentation might say. @@ -197,15 +198,18 @@ -- -- /See:/ 'record' smart constructor. data Record = Record' - { _rSequenceNumber :: !Text - , _rData :: !Base64 - , _rPartitionKey :: !Text + { _rApproximateArrivalTimestamp :: !(Maybe POSIX) + , _rSequenceNumber :: !Text + , _rData :: !Base64 + , _rPartitionKey :: !Text } deriving (Eq,Read,Show,Data,Typeable,Generic) -- | Creates a value of 'Record' with the minimum fields required to make a request. -- -- Use one of the following lenses to modify other fields as desired: -- +-- * 'rApproximateArrivalTimestamp' +-- -- * 'rSequenceNumber' -- -- * 'rData' @@ -218,19 +222,25 @@ -> Record record pSequenceNumber_ pData_ pPartitionKey_ = Record' - { _rSequenceNumber = pSequenceNumber_ + { _rApproximateArrivalTimestamp = Nothing + , _rSequenceNumber = pSequenceNumber_ , _rData = _Base64 # pData_ , _rPartitionKey = pPartitionKey_ } --- | The unique identifier for the record in the Amazon Kinesis stream. +-- | The approximate time that the record was inserted into the stream. +rApproximateArrivalTimestamp :: Lens' Record (Maybe UTCTime) +rApproximateArrivalTimestamp = lens _rApproximateArrivalTimestamp (\ s a -> s{_rApproximateArrivalTimestamp = a}) . mapping _Time; + +-- | The unique identifier of the record in the stream. rSequenceNumber :: Lens' Record Text rSequenceNumber = lens _rSequenceNumber (\ s a -> s{_rSequenceNumber = a}); -- | The data blob. The data in the blob is both opaque and immutable to the -- Amazon Kinesis service, which does not inspect, interpret, or change the --- data in the blob in any way. The maximum size of the data blob (the --- payload before base64-encoding) is 50 kilobytes (KB) +-- data in the blob in any way. When the data blob (the payload before +-- base64-encoding) is added to the partition key size, the total size must +-- not exceed the maximum record size (1 MB). -- -- /Note:/ This 'Lens' automatically encodes and decodes Base64 data, -- despite what the AWS documentation might say. @@ -249,8 +259,10 @@ = withObject "Record" (\ x -> Record' <$> - (x .: "SequenceNumber") <*> (x .: "Data") <*> - (x .: "PartitionKey")) + (x .:? "ApproximateArrivalTimestamp") <*> + (x .: "SequenceNumber") + <*> (x .: "Data") + <*> (x .: "PartitionKey")) -- | The range of possible sequence numbers for the shard. --