AWS re:Invent - Largest developer conference in the world - Part Two

Following part one of our blog post, we're continuing to explore the innovative products across container tech, to serverless applications. We'll start by reviewing Fargate...

Fargate

This is container tech that allows you to spin up just a container, and it will run it. You don’t have to manage EC2 instances underneath. This is what I hoped ECS was going to be when it was announced. It can work with the existing ECS service /or/ more excitingly for us — can be used with their managed Kubernetes service EKS. Swit swoo.There's a really helpful introduction to Fargate from AWS direct, you can read that "here":https://aws.amazon.com/blogs/aws/aws-fargate/.

Serverless Application Repository

This is a place where you can use ready-to-go applications built by other people. You can also share applications with your team, or make them public. All public apps must include source code, and you can’t bill people from within here. If you’re selling SaaS this could be ideal — an example app with your free-tier API usage baked in, with a simple way for them to upgrade.It uses the Serverless Application Model; think of it as a kind of simplified CloudFormation (I believe it actually compiles to CF underneath — if you want to go into depth). This is what Serverless Framework was solving early on, though they have now expanded to multi-cloud and further into analytics.See our blog post on "Severless Scheduled Tasks here":https://parall.ax/blog/view/3202/tutorial-serverless-scheduled-tasks.

New Spot Pricing Model

This brings spot pricing to the masses without needing to understand bidding or spot markets. You can simply spin up an instance using the regular API, and pay up to 90% less for it. It uses predictable prices that adjust gradually based on long-term trends. If you have non-time-sensitive batch processing workload, that for some reason you’re still running on an EC2 fleet, this could be a great way to eek some cost savings without much effort.

Aurora Serverless

This could be an absolute game changer. Serverless applications currently suffer when trying to find a suitable data layer. Spinning up a server to handle a database kind of defeats the object. Shoehorning data into DynamoDB just because of its pricing model seems counter-intuitive.I can see this being great for low volume sites or unpredictable workloads. We run a lot of traffic-heavy campaigns that we currently use DynamoDB for, but this might make sense instead. We’ll do some testing when we get our mitts on it.Another advantage is HTTP API access for querying, which can make it perfect for a microservice.

DynamoDB Backup and Restore

Finally! This was always a royal PITA. There was a way to do it with data pipelines but was often too slow or clunky to set up.We’ve used a Python package called dynamodump, which worked well enough. Until it didn’t. It scales up the Read/Write capacity then puts it back down again depending on if you’re doing a backup or restore. There’s a limit to how many times you can do this in a period if you’re doing some serious data wrangling you can quickly get caught out.I’m so glad this is a feature now. Will be giving this one a whirl as soon as I get back into the office.

DynamoDB Global Tables

Write once, read anywhere. No this isn’t a Java advert. If you were using DynamoDB for a global mobile app or web application, you might have been doing some significant wrangling and jiggery-pokery to get the performance characteristics you want. Or you just give up and run in a single region. Now you can just click a tick box, (presumably pay multiple times more) and get a table available in each of those regions. Pretty impressive stuff.

Neptune

This a graph database, which is great at a certain class of problems such as recommendation engines, fraud detection, and social graphs.For smaller apps, you could almost get away with doing this within a Relational Database, but then you can start running into performance issues. Neptune looks like the one if you’ve got a larger dataset.

AWS Cloud9

Aha, I had a feeling they might save this for the keynote. This has been lurking in plain sight for a while.It used to be Bespin which was an editor started at Mozilla by Dion Almaer. He was also the guy that ran Ajaxian.com (remember that!?) It’s a cloud-based IDE that spins up your own EC2 instance. The instance winds down so you don’t get charged when you’re not using it.The killer feature here is breakpoint debugging for Lambda functions. If this works as well as advertised it could be an absolute game-changer.

Amazon FreeRTOS

This is very interesting. Though the parts specific to Amazon are less so for our current projects.FreeRTOS is an open source Real-time Operating System for Microcontrollers.In massively memory constrained and CPU constrained environments, it doesn’t make sense to run a full-blown OS. This takes away some of that pain by providing lightweight memory management, queues, and a whole bunch of Public Key Cryptography stuff, including hashing and verification.The AWS IoT integration part is useless for LoRaWAN as it operates over TLS, which is way too much to then put over LoRaWAN. I don’t think it makes much sense tying yourself to Amazon at this layer. You’d do it higher up.

IoT 1-click

A product born out of AWS staffers doing demos I imagine. This allows you to map the pressing of a hardware button to a Lambda function. Great for demos, loses most of its value for pretty much anything else I imagine.

IoT analytics

Again this looks like a solution looking for a problem potentially. It does mention who IoT can have a lot of missing, fuzzy and incorrect data. How it helps you manage and fix this is less clear. There’s a time series database in there too. Talking to people at the conference, they would usually just use Kinesis Streams to ingest data, then process it like you would any other. There’s nothing ‘IoT’ specific about the problem I don’t think.

IoT Device Defender

I need to learn more about this, as the details are quite sketchy at the moment. It appears to be some kind of security data collector. It can detect data coming from unauthorised endpoints. This does seem something you’d build into your app anyway, so not sure of the specific advantages of using this. Would be interested to hear what other customers are doing with it and where the need arose.It can emit SNS and CloudWatch events.

IoT Device Manager

This sounded interesting at first, but then quickly became boring as I read through the docs. The advertising spiel says it can do updates, patching, factory resets. What it actually means is it can emit jobs, and you can call them what you want, ‘reboot’ for example, then handle that on your own side. There’s nothing that helps you do any of that.What it does appear to do well is set device certificates and apply labels to devices. If you’re using IoT core / greengrass on the device is interesting, if you’re not - then it’s not. :-(

Greengrass ML Inference

The inference is just the detection part or inferring the outcome from a premade prediction model. This new product allows you to build a model in the cloud (using SageMaker for example) then optimise it to run on a local device; even if that device doesn’t have access to the internet.The service understands how to compress and simplify the model so it can run within the power constraints. It looks like this is what Amazon is using to power its DeepLens product. This is a small Intel computer with a camera attached.Please "read our third and final blog post":https://parall.ax/blog/view/3259/aws-re-invent-largest-developer-conference-in-the-world-part-three focusing on Amazon Comprehend including recognition for video, SageMaker and some other engaging AWS talking points...