Tag Archives: Big Data

Technology Highlights Google Cloud Next 2017

Google cloud next-2017 The largest Google developer and IT gathering in Amsterdam to explore the latest developments in cloud technology. A chance to engage with the foremost minds leading the cloud revolution and learn how the modern enterprise is benefiting from the latest in cloud technology in unprecedented ways. As usual for us one more way to keep up with technology.

We saw some very interesting new innovations (spanner and app maker to name two) and how they relate to application development in the cloud.

Given below are the other highlights of the technologies talked about during the event:

1) Microservices & Kubernetes:

Microservices – is an architectural style that structures an application as a collection of loosely coupled services, which implement business capabilities. It enables the continuous delivery/deployment of large, complex applications and enables an organization to evolve its technology stack and can develop and deploy faster. It’s an evolution of software development and deployment that embraces DevOps and containers and breaks applications down to smaller individual components.

The emerging combination of micro-service architectures, Docker containers, programmable infrastructure, cloud, and modern Continuous Delivery (CD) techniques have enabled a true paradigm shift for delivering business value through software development.

The combination of microservices and containers promotes a totally different vision of how services change application development.

Kubernetes – is an open-source system for automating deployment, scaling and management of containerized applications that was originally designed by Google and donated to the cloud native computing foundation. It aims to provide a “platform for automating deployment, scaling, and operations of application containers across clusters of hosts”. It supports a range of container tools, including Docker.

Technology Highlights in Google Cloud Next 2017

2) Choosing the right compute option in a cloud project: a decision tree

To understand the trade-offs and decide which models are the best fit for your systems as well as how the models map to Cloud services —Compute Engine, Container Engine, App Engine, cloud functions.

Compute Engine is an Infrastructure-as-a-Service. The developer has to create and configure their own virtual machine instances. It gives them more flexibility and generally costs much less than App Engine. The drawback is that the developer has to manage their app and virtual machines yourself.

Container Engine is another level above Compute Engine, i.e. it’s cluster of several Compute Engine instances which can be centrally managed.

App Engine is a Platform-as-a-Service. It means that the developer can simply deploy their code, and the platform does everything else for them.

Cloud Functions is a serverless computing service, the next level up from App Engine in terms of abstraction. It allows developers to deploy bite-size pieces of code that execute in response to different events, which may include HTTP requests, changes in Cloud Storage, etc.

Technology Highlights in Google Cloud Next 2017

3) Big data – Big data refers to data that would typically be too expensive to store, manage, and analyse using traditional (relational and/or monolithic) database systems. Usually, such systems are cost-inefficient because of their inflexibility for storing unstructured data (such as images, text, and video), accommodating “high-velocity” (real-time) data, or scaling to support very large (petabyte-scale) data volumes. There are new approaches to managing and processing big data, including Apache Hadoop and NoSQL database systems. However, those options often prove to be complex to deploy, manage, and use in an on-premise situation.

Cloud computing offers access to data storage, processing, and analytics on a more scalable, flexible, cost-effective, and even secure basis than can be achieved with an on-premise deployment.

Big Data with SAP HANA

I’ve been interested in large scale computing ever since I was introduced to it at the University of Southampton where the Computer Science department was heavily involved in Data Mining and Grid Computing research, which obviously influenced the courses on offer and what the lecturers liked to talk about. My dissertation looked at how these techniques could be applied to protein folding research, which generated much more data than could be handled effectively with the technologies available at the time. We are producing more and more data at an ever increasing rate each day, so these challenges are becoming relevant to more and more businesses and sectors. Of course things have moved on in recent years. The buzz words today are Cloud Computing and Big Data, evolutions on those older concepts that I’ve followed with great interest.

Given this I was happy to recently have the chance to do some research into SAP’s in-memory database solution HANA (High-performance Analytic Appliance, a somewhat tortured acronym). Clearly it’s a complex platform and I’ve only scratched the surface of what it’s capable of. It’s tricky to sum up, but to quote from a SAP HANA white paper (http://www.saphana.com/blogs/experts/2012/02/01/latest-sap-hana-whitepaper-reveals-more-than-just-technical-details):

“The SAP HANA® database is an in-memory database that combines transactional data processing, analytical data processing, and application logic processing functionality in memory. SAP HANA removes the limits of traditional database architecture that have severely constrained how business applications can be developed to support real-time business.”

This blog post is a good place to start if you would like to know more about SAP HANA: http://www.saphana.com/community/blogs/blog/2012/09/07/sap-hana-for-beginners

One of the benefits that really jumped out at me was the ability to query for complex aggregated data in real time. With a standard database it can take too much time to fetch, for example, the balance of a financial ledger. Because of this it’s left to the application layer to run batch jobs which calculate these figures and store them in additional tables. HANA allows you to delegate this task back to the database layer. We can therefore simplify our data models, we don’t have to write specific application code to handle the task and our totals are truly “real time” because we do not have to run nightly batch jobs to update them. This leaves our application to focus on what it should be doing, implementing business logic and not worrying about how to overcome the limitations of the database platform.

Also of note is that HANA provides a standard SQL interface over ODBC, this means that existing applications can start to use its functionality with a minimum of migration effort. In businesses utilising HANA, Uniface would be able to play a crucial role in tying together various different applications and technologies they might have. One use case might be for a Uniface interface able to pump real time data into HANA from other disparate applications. Since a HANA database is likely to be structure differently to a traditional database Uniface’s powerful ability to transform and map data would be invaluable to the process.

It’s fairly simple to try out HANA. SAP provides a very handy 30 day developer trial on CloudShare (http://scn.sap.com/docs/DOC-28191). Once you’re signed up it’s just a case of logging into CloudShare and then starting up the VM environment. You’re then free to do whatever you like with the servers. In my case I uploaded a copy of Uniface, ran the ODBC configuration utility to setup my driver settings and had a play around. Incidentally, I can highly recommend taking an initial snapshot of your VMs so that you can revert to it in case you completely break the servers (like I did).

Banking on Apps (Part 2)

Guest contributor, Paul Herzlich, Principal Analyst, Creative Intellect Consulting

Read part 1 here

Bruisers and browsers: through thick and thin

Another perennial seesaw act in software architectures has been the thin or thick client debate.

With the Internet came the viability of the thin client (the concept had been around for many years, after all, what were dumb terminals?). However, permanent connectivity and bandwidth have made it a fairly slow process of moving towards computing in what we now call the Cloud. Let’s take email as an example. It took quite a few years for people to accept using web mail solutions instead of Outlook or other desktop mail client. And to this day, thick mail clients are still widely used. Likewise for most ‘office productivity applications’. Maybe that is now changing.

But the Internet did usher in a whole new era of self-service applications like banking, retail and even financial trading. These new applications were presented to us through the browser, which became a powerful execution and presentation engine through the addition of Web 2.0 technologies. It is now to the point where a browser-based application can be as functional, interactive, responsive and robust (maybe more so) as a thick client application.

The heterogeneity of technologies to achieve this is both a wonder and a worry. Look under the hood and you find a dog’s dinner of simple HTML with embedded scripts – in multiple script languages; outward references to style sheets who are acting in a passive, declarative way; calls to external services; coupled with implicit browser behaviour. Who’s in charge? Use CSS floats and you can’t even tell what order the HTML elements will be displayed on the page. Not the worst problem in the world, but a fundamental violation of a simple premise of HTML in the beginning that things just stacked on top of each other from top to bottom.

Never mind where’s the data? Where’s the program? It’s all in there somewhere. Despite this muddlement, the browser became a bruiser and looked destined to dominate the shape of programs and programming until…

Back to the smartphone future

What got me thinking about data versus programs and thin versus thick clients – about architectures in general –  was the introduction of the iPhone. Among the many disruptions it caused, it equivocated over the direction of application development. At the launch, Steve Jobs proudly revealed how browsing on the iPhone became feasible. The device had a clear screen. Tap to enlarge an area of interest on a page. Touch to interact with buttons, good drop down lists, usable text entry. Anything you could work with in a browser on your desktop computer, you could do on the iPhone. (Except Flash, we all know.)

But in the wake of the phenomenal success of the iTunes Store, Apple also launched the App Store, which has become a model for the industry. What we got from that was a whole new way (yet again) of delivering old-fashioned programs. I don’t know whether Apple knew how this would pan out. But what has happened drives me to blog. Or should I say rant?

Rant 1: Simple. Rejoice. Great to have new ways of delivering functionality that requires portability and mobility.

Rant 2: Democratisation of software. Anyone with an idea can build and market an App. They could have done that on the Internet using web technologies. The App Store adds the dimension of a clear marketing vehicle. It was probably more rewarding when the Apps numbered in the thousands. Trouble is, of course, app stores (no brand intended here) give credibility to some unholy and unmitigated junk. I resent the lies, the deceptively named-to-confuse Apps, and the debilitated functionality. This includes software that is nothing more than advertising. How does it get through screening? As bad as a link farm.

Rant 3: Half-baked mobile software. Make up your minds. Is it something I should run in a browser or is it a self-contained App. Thank you for your App’s big buttons to select a primary function, but shame on you for those buttons that simply take me to an in-App or out-of-App browser window. The user interface suddenly changes and the out-of-App browser window has no integral navigation back to the App.

Rant 4: If I have the App please don’t run the browser version. I hate when this happens: Facebook, YouTube, Twitter – Apps all on my iPhone. Sometimes they open in the browser; sometimes in the App. Why?

Rant 5: Same functionality everywhere, please. I notice that with iOS 6, if you use the browser it will tell you that there is an App. Good idea. I’ll get that App. Of course, your App had better not do any of the nonsense mentioned in my prior three themes, and one other further thing, too. If you tell me to use the App, it had better have at least the same functionality as your browser application. The worst case is that they both do some things not done by the other. That’s a nightmare for a user – for your customers if you need reminding.

Rant 6: Where’s my data? Clearly not an issue for most people, although I daresay, some people might detect there’s something wrong when they can’t see text done in one program (say a Note) when they want to look at it in another program (say a Diary). The trouble is that programs make very poor viewers of collections of data. If I have word processing documents filed by topic on my computer, why can’t I have them filed with documents of all formats about the same topic on my smartphone or tablet? Is staying organized really anathema to mobile computing? I know that Search is the new organizational paradigm (i.e. don’t organize, but instead use Search find your needle in a haystack) but I just did an experiment to see if I can find a Pages document via title or contents on my iPhone with Spotlight. The answer is no.

In Pursuit of Sustainable Legacy Modernization (Part 1)

Guest contributor, Paul Herzlich from analyst firm Creative Intellect Consulting

The term ‘modernization’ is one of those buzzwords that gain a lot of currency at a particular moment. Clearly the idea is not new. We’ve always had to maintain and upgrade our systems to keep them ‘up-to-date’. In the 1980’s software conversion or migration was a popular solution. In the 1990’s, it was often termed ‘re-engineering’. What more do we mean by the term modernization?

A few things stand out:

    • The internet for global access was a revolution that has made it possible to reach customers, employees and business partners directly and at low cost. The resulting boon in self-service applications, access anywhere and shared services has eliminated huge business processing costs.
    • New technologies – mobile devices and Big Data are examples – make it possible to offer new products and services and to manage your business more effectively.
    • New paradigms for client and employee engagement – like social media, collaboration and app store distribution – are expected for improved communication, ease of contribution and democratized participation.
    • Cloud deployment can transform operating costs and bring new levels of speed, flexibility and responsiveness to IT service provision.

Automated support for your core back and front office business processes alone is no longer sufficient. Cutting edge software and platforms are ‘must haves’ for meeting and outstripping the competition. Today’s modernization is not only a software imperative but also a business one.

The industry knows this but there’s a twist. Although discussions of modernization often center on a world of technically dazzling possibilities, the reality for many organizations is much less exciting.