The goal of the “Long Ride Alerts” exercise is to indicate whenever a taxi ride started two hours ago, and is still ongoing.
This can be done fairly straightforwardly with a
ProcessFunction (see the Long Ride Alerts exercise), but this requires you to explicitly manage state and timers. In this exercise we’ll do this more simply using Flink’s CEP library.
Using the CEP library requires adding these dependencies to the
pom.xml of your Maven project:
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-cep_2.11</artifactId> <version>1.4.0</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-cep-scala_2.11</artifactId> <version>1.4.0</version> </dependency>
The input data of this exercise is a
DataStream of taxi ride events. You will want to use a
DataStream<TaxiRide> rides = env.addSource( new CheckpointedTaxiRideSource(input, servingSpeedFactor));
Even when used with a delay factor of zero, the
TaxiRideSource that you may’ve used before will reorder events with the same timestamp, and that adds complexity that we’d rather avoid for now.
Don’t bother trying to filter the events (as is done in the Taxi Ride Cleansing exercise).
The result of the exercise should be a
DataStream<TaxiRide> that only contains START events of taxi rides which have no matching END event within the first two hours of the ride.
The resulting stream should be printed to standard out.
The expected results are something like this:
Here are the rideIds and start times of the first few rides that go on for more than two hours, but you might want to print other info as well:
934753,2013-01-01 01:03:00 950600,2013-01-01 01:14:00 952469,2013-01-01 01:53:00 946984,2013-01-01 02:21:00 958062,2013-01-01 02:26:00 1029600,2013-01-01 10:51:00 ...
Reference solutions are available at GitHub: