Skip to content

Commit f206611

Browse files
committed
Merged in SUMOK-8 (pull request #2)
SUMOK-8 SUMOK-9 Added unit tests and README file
2 parents 4d44e79 + bbd4305 commit f206611

37 files changed

Lines changed: 1143 additions & 836 deletions

README.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# Kinesis-Sumologic Connector
2+
3+
The **Kinesis-Sumologic Connector** is a Java connector that acts as a pipeline between an [Amazon Kinesis] stream and a [Sumologic] Collection. Data gets fetched from the Kinesis Stream, transformed into a POJO and then sent to the Sumologic Collection as JSON.
4+
5+
## Requirements
6+
7+
+ [Java JDK 1.7]: This connector has been built with Java version 1.7.
8+
+ [Ant]: A build.xml script has been provided to build the connector with Ant.
9+
+ [AWS Kinesis Account]: An Amazon AWS Kinesis account to use as a source of data.
10+
+ [Sumologic Account]: A Sumologic account to use as a destination.
11+
12+
## Overview
13+
14+
Incoming records from one (or many) Shards of an AWS Kinesis Stream will be read using the [Kinesis Client Library]. Records will be:
15+
16+
+ **Transformed**: Raw records will be transformed into a POJO using the KinesisMessageModel class and then serialized.
17+
+ **Filtered**: A filter may be applied to the records. Default filter will let all records pass.
18+
+ **Buffered**: A custom buffer may be used to define thresholds that, when crossed, will flush all records into the emitter.
19+
+ **Emitted**: The records will get send to the Sumologic Collector.
20+
21+
## Configuration
22+
23+
A sample properties file is provided, which should be modified to use your AWS Accounts (**accessKey** and **secretKey**), Kinesis Stream(**kinesisInputStream**), Sumologic HTTP source (**sumologicUrl**) and App Name (**appName**).
24+
25+
## Running the Connector
26+
27+
After modifying the .properties file, run the connector by executing the ant build script
28+
To download the needed dependencies execute **ant setup**
29+
To build and execute the connector execute **ant run**
30+
31+
32+
## Related sources
33+
34+
[Amazon Kinesis](http://aws.amazon.com/kinesis/)
35+
36+
[Sumologic](https://www.sumologic.com/)
37+
38+
[Java JDK 1.7](http://www.oracle.com/technetwork/java/javase/overview/index.html)
39+
40+
[Ant](http://ant.apache.org/)
41+
42+
[AWS Kinesis Account](http://aws.amazon.com/account/)
43+
44+
[Sumologic Account](https://www.sumologic.com/pricing/)
45+
46+
[Kinesis Client Library](https://github.com/awslabs/amazon-kinesis-client/)

build.xml

Lines changed: 41 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,6 @@
1313
<fileset dir="${external.dir}" includes="**/*.jar" />
1414
<pathelement location="${basedir}/build" />
1515
<pathelement location="." />
16-
<pathelement location="${basedir}/samples" />
17-
<pathelement location="${basedir}/samples/sumologic" />
1816
</path>
1917

2018
<target name="clean">
@@ -44,10 +42,48 @@
4442
<antcall target="download"/>
4543
<antcall target="unzip"/>
4644
</target>
47-
48-
<target name="run">
45+
46+
<target name="test">
47+
<mkdir dir="${basedir}/build"/>
48+
<javac srcdir="${src.dir}/main/java"
49+
destdir="${basedir}/build"
50+
classpathref="sumologic.classpath"
51+
includes="com/mcplusa/kinesis/*.java, com/mcplusa/kinesis/utils/*.java,
52+
com/mcplusa/sumologic/*.java, com/mcplusa/sumologic/implementations/*.java"/>
53+
<javac srcdir="${src.dir}/test/java"
54+
destdir="${basedir}/build"
55+
classpathref="sumologic.classpath"
56+
includes="com/mcplusa/kinesis/*.java, com/mcplusa/kinesis/utils/*.java,
57+
com/mcplusa/sumologic/*.java, com/mcplusa/sumologic/implementations/*.java"/>
58+
<junit haltonfailure="yes"
59+
showoutput="yes"
60+
fork="yes"
61+
errorProperty="test.failed"
62+
failureProperty="test.failed">
63+
<classpath refid="sumologic.classpath"/>
64+
<formatter
65+
type="brief"
66+
usefile="false"/>
67+
68+
<batchtest fork="yes" todir="${reports.tests}">
69+
<fileset dir="${src.dir}/test/java">
70+
<include name="**/*Test*.java"/>
71+
</fileset>
72+
</batchtest>
73+
</junit>
74+
</target>
75+
76+
<target name="main">
4977
<mkdir dir="${basedir}/build"/>
50-
<javac srcdir="${src.dir}/" destdir="${basedir}/build" classpathref="sumologic.classpath" />
78+
<javac srcdir="${src.dir}/main/java"
79+
destdir="${basedir}/build"
80+
classpathref="sumologic.classpath"
81+
includes="com/mcplusa/kinesis/*.java, com/mcplusa/kinesis/utils/*.java,
82+
com/mcplusa/sumologic/*.java, com/mcplusa/sumologic/implementations/*.java"/>
5183
<java classname="com.mcplusa.sumologic.SumologicExecutor" classpathref="sumologic.classpath" fork="true" />
84+
</target>
85+
86+
<target name="run">
87+
<antcall target="main"/>
5288
</target>
5389
</project>

external/lib/hamcrest-core-1.3.jar

44 KB
Binary file not shown.

external/lib/wiremock-1.56.jar

220 KB
Binary file not shown.

external/test/guava-18.0.jar

2.15 MB
Binary file not shown.
93.6 KB
Binary file not shown.

external/test/jetty-6.1.26.jar

527 KB
Binary file not shown.
173 KB
Binary file not shown.

external/test/jopt-simple-4.9.jar

64.9 KB
Binary file not shown.

external/test/json-path-2.0.0.jar

133 KB
Binary file not shown.

0 commit comments

Comments
 (0)