Amazon Kinesis Data Analytics Developer Guide 2.Dev
User Manual:
Open the PDF directly: View PDF
Page Count: 296 [warning: Documents this large are best viewed by clicking the View PDF Link!]
- Amazon Kinesis Data Analytics
- Table of Contents
- What Is Amazon Kinesis Data Analytics?
- Amazon Kinesis Data Analytics: How It Works
- Configuring Application Input
- Configuring a Streaming Source
- Configuring a Reference Source
- Working with JSONPath
- Mapping Streaming Source Elements to SQL Input Columns
- Using the Schema Discovery Feature on Streaming Data
- Using the Schema Discovery Feature on Static Data
- Preprocessing Data Using a Lambda Function
- Parallelizing Input Streams for Increased Throughput
- Application Code
- Configuring Application Output
- Creating an Output Using the AWS CLI
- Using a Lambda Function as Output
- Delivery Model for Persisting Application Output to an External Destination
- Error Handling
- Granting Amazon Kinesis Data Analytics Permissions to Access Streaming and Reference Sources (Creating an IAM Role)
- Trust Policy
- Permissions Policy
- Permissions Policy for Reading an Kinesis Stream
- Permissions Policy for Reading a Kinesis Data Firehose Delivery Stream
- Permissions Policy for Writing to a Kinesis Stream
- Permissions Policy for Writing to a Firehose Delivery Stream
- Permissions Policy for Reading a Reference Data Source from an Amazon S3 Bucket
- Automatically Scaling Applications to Increase Throughput
- Configuring Application Input
- Getting Started with Amazon Kinesis Data Analytics
- Streaming SQL Concepts
- Example Applications
- Examples: Transforming Data
- Preprocessing Streams with Lambda
- Examples: Transforming String Values
- Example: Extracting a Portion of a String (SUBSTRING Function)
- Example: Replacing a Substring using Regex (REGEX_REPLACE Function)
- Example: Parsing Log Strings Based on Regular Expressions (REGEX_LOG_PARSE Function)
- Example: Parsing Web Logs (W3C_LOG_PARSE Function)
- Example: Split Strings into Multiple Fields (VARIABLE_COLUMN_LOG_PARSE Function)
- Example: Transforming DateTime Values
- Example: Transforming Multiple Data Types
- Examples: Windows and Aggregation
- Examples: Joins
- Examples: Machine Learning
- Examples: Alerts and Errors
- Examples: Solution Accelerators
- Examples: Transforming Data
- Monitoring Amazon Kinesis Data Analytics
- Monitoring Tools
- Monitoring with Amazon CloudWatch
- Kinesis Data Analytics Metrics and Dimensions
- Viewing Amazon Kinesis Data Analytics Metrics and Dimensions
- Creating CloudWatch Alarms to Monitor Amazon Kinesis Data Analytics
- Working with Amazon CloudWatch Logs
- Limits
- Best Practices
- Troubleshooting Amazon Kinesis Data Analytics
- Unable to Run SQL Code
- Unable to Detect or Discover My Schema
- Reference Data is Out of Date
- Application Not Writing to Destination
- Important Application Health Parameters to Monitor
- Invalid Code Errors When Running an Application
- Application is Writing Errors to the Error Stream
- Insufficient Throughput or High MillisBehindLatest
- Authentication and Access Control for Amazon Kinesis Data Analytics
- Authentication
- Access Control
- Overview of Managing Access Permissions to Your Amazon Kinesis Data Analytics Resources
- Using Identity-Based Policies (IAM Policies) for Amazon Kinesis Data Analytics
- Permissions Required to Use the Amazon Kinesis Data Analytics Console
- AWS Managed (Predefined) Policies for Amazon Kinesis Data Analytics
- Customer Managed Policy Examples
- Step 1: Create an IAM User
- Step 2: Allow the User Permissions for Actions that Are Not Specific to Amazon Kinesis Data Analytics
- Step 3: Allow the User to View a List of Applications and View Details
- Step 4: Allow the User to Start a Specific Application
- Step 5: Allow the User to Create an Amazon Kinesis Data Analytics Application
- Step 6: Allow the Application to Use Lambda Preprocessing
- Amazon Kinesis Data Analytics API Permissions: Actions, Permissions, and Resources Reference
- Kinesis Data Analytics SQL Reference
- API Reference
- Actions
- AddApplicationCloudWatchLoggingOption
- AddApplicationInput
- AddApplicationInputProcessingConfiguration
- AddApplicationOutput
- AddApplicationReferenceDataSource
- CreateApplication
- DeleteApplication
- DeleteApplicationCloudWatchLoggingOption
- DeleteApplicationInputProcessingConfiguration
- DeleteApplicationOutput
- DeleteApplicationReferenceDataSource
- DescribeApplication
- DiscoverInputSchema
- ListApplications
- StartApplication
- StopApplication
- UpdateApplication
- Data Types
- ApplicationDetail
- ApplicationSummary
- ApplicationUpdate
- CloudWatchLoggingOption
- CloudWatchLoggingOptionDescription
- CloudWatchLoggingOptionUpdate
- CSVMappingParameters
- DestinationSchema
- Input
- InputConfiguration
- InputDescription
- InputLambdaProcessor
- InputLambdaProcessorDescription
- InputLambdaProcessorUpdate
- InputParallelism
- InputParallelismUpdate
- InputProcessingConfiguration
- InputProcessingConfigurationDescription
- InputProcessingConfigurationUpdate
- InputSchemaUpdate
- InputStartingPositionConfiguration
- InputUpdate
- JSONMappingParameters
- KinesisFirehoseInput
- KinesisFirehoseInputDescription
- KinesisFirehoseInputUpdate
- KinesisFirehoseOutput
- KinesisFirehoseOutputDescription
- KinesisFirehoseOutputUpdate
- KinesisStreamsInput
- KinesisStreamsInputDescription
- KinesisStreamsInputUpdate
- KinesisStreamsOutput
- KinesisStreamsOutputDescription
- KinesisStreamsOutputUpdate
- LambdaOutput
- LambdaOutputDescription
- LambdaOutputUpdate
- MappingParameters
- Output
- OutputDescription
- OutputUpdate
- RecordColumn
- RecordFormat
- ReferenceDataSource
- ReferenceDataSourceDescription
- ReferenceDataSourceUpdate
- S3Configuration
- S3ReferenceDataSource
- S3ReferenceDataSourceDescription
- S3ReferenceDataSourceUpdate
- SourceSchema
- Actions
- Document History for Amazon Kinesis Data Analytics
- AWS Glossary