Welcome!

Big Data Journal Authors: Dana Gardner, Liz McMillan, Kira Makagon, Elizabeth White, Pat Romanski

Blog Feed Post

Well Engineered use of AWS by Recovery and Transparency Board (RATB)

By

After speaking with Shawn Kingsberry in preparations for our 4 April Government Big Data Forum I realized their use of Amazon Web Services (AWS) may be of very high interest to our readers and went about looking for more info online to see what was publicly available. I was ecstatic to see a well written use case for much of it is on the AWS website. That write-up includes a nice graphic that is helpful to understanding how things were done.

Since this is provided by AWS as a way of articulating their special contributions it does not go into the many other services and components required to make this work. But many of those components are probably modular and exchangable with other capabilities. So this overview is probably a great way to get a baseline on what the RATB architecture is.

With that, the following is from: http://aws.amazon.com/solutions/case-studies/ratb/

AWS Case Study: Recovery.gov and AWS Bring Transparency to the Cloud

The Recovery Accountability and Transparency Board (RATB) was established when Congress passed the American Recovery and Reinvestment Act (ARRA) in February, 2009. To ensure against waste, fraud, and abuse, the RATB was tasked with developing a Website which met the following goals:

  • Provide easily accessible information to the public on Recovery spending and results
  • Promote official data in public debate
  • Provide fair and open access to Recovery opportunities
  • Enable public accountability for Recovery spending
  • Promote an understanding of the local impact of Recovery spending

The resulting Website is Recovery.gov.

The RATB originally intended to use Amazon Web Services (AWS) only for development, testing, and as failover, but, says Jim Warren, RATB Chief Information Officer, “When AWS outperformed our on-premises solution at a fraction of the cost, the prime contractor Smartronix and its lead sub-contractor Synteractive, provided a compelling justification for the RATB to host Recovery.gov on AWS’s platform.”

According to Mr. Warren, Smartronix selected AWS because of the flexibility provided by AWS’s Infrastructure as a Service (IaaS) model; track record of providing infrastructure for large-scale commercial projects; focus on cost-effectiveness and a pay-as-you-go-model that allowed Smartronix to control costs; commitment to security and reliability; and its FISMA Low certification.

The RATB now uses the following AWS services: Amazon Elastic Compute Cloud (Amazon EC2), Amazon Simple Storage Service (Amazon S3), Amazon Elastic Block Storage (Amazon EBS), Elastic Load Balancing (ELB), and Amazon CloudWatch. The solution also combined multiple pieces of software.

The following diagrams illustrate their topology:

 

ratb arch diagram Well Engineered use of AWS by Recovery and Transparency Board (RATB)

Recovery Accountability and Transparency Board

Business Intelligence and Data Warehousing
The website uses Microsoft’s SharePoint as it content management system and all data is aggregated into a global dimensional data warehouse to facilitate time-based analysis and reporting. The solution leverages SAP BusinessObjects and Microsoft SQL Server for reporting services that show how and where the money is being spent. The BI tools enable ad hoc reporting and are instrumental in Data Quality and Data Integrity score-carding.

Advanced Geospatial Analysis and Mapping
The Geospatial tools, based on ESRI software, allow up to 5,000 concurrent users and enables them to go directly to go to their communities of interest at the state, zip, congressional district, or county level. Hundreds of thousands of addresses are geo-coded and aggregated to display total value for each area of interest. Thematic maps and multiple view selections were incorporated to help the user better visualize the data. These thematics include funding heat maps, unemployment heat maps, and diversity maps.

Mr. Warren notes that testing and development enclaves were procured and ready on Amazon EC2 within two days of the contract award. He says, “Our migration to the cloud took only 22 days from feasibility study to production.” The RATB has also enjoyed improved computer security, including greater protection against network attacks and real-time detection of system tampering. Mr. Warren says, “In essence, the security system of AWS’s platform has been added to our existing security systems. We now have a security posture consistent with that of a multi-billion dollar company.” Additional benefits include lower costs and ability to add capacity on demand. The RATB expects to save around $750K during their current budget cycle.

The success of Recovery.gov is being noticed outside of the RATB as well: Andre Romano of Newsweek wrote, “The current incarnation of Recovery.gov…is perhaps the clearest, richest interactive database ever produced by the American bureaucracy.” The site has been given the 2009 Merit award, the 2010 Gold Addy award for Website design, InformationWeek Government IT Innovator 2010 Award, an Award of Distinction during the 16th Annual Communicator Awards, and a second place Gold Screen Award from the National Association of Government Communicators. Recovery.gov is also an official Honoree for the Financial Services category in the 14th Annual Webby Awards.

To learn more see http://recovery.gov

 Well Engineered use of AWS by Recovery and Transparency Board (RATB)

Read the original blog entry...

More Stories By Bob Gourley

Bob Gourley, former CTO of the Defense Intelligence Agency (DIA), is Founder and CTO of Crucial Point LLC, a technology research and advisory firm providing fact based technology reviews in support of venture capital, private equity and emerging technology firms. He has extensive industry experience in intelligence and security and was awarded an intelligence community meritorious achievement award by AFCEA in 2008, and has also been recognized as an Infoworld Top 25 CTO and as one of the most fascinating communicators in Government IT by GovFresh.

Latest Stories from Big Data Journal
Come learn about what you need to consider when moving your data to the cloud. In her session at 15th Cloud Expo, Skyla Loomis, a Program Director of Cloudant Development at Cloudant, will discuss the security, performance, and operational implications of keeping your data on premise, moving it to the cloud, or taking a hybrid approach. She will use real customer examples to illustrate the tradeoffs, key decision points, and how to be successful with a cloud or hybrid cloud solution.
For the last hundred years, the desk phone has been a staple of every business. The landline has been a lifeline to customers and colleagues as the primary means of communication – even as email threatened to render the telephone obsolete. For some purposes, like conference calling, there was simply no substitute. That is, until a few years ago. With all due respect and apologies to Mr. Alexander Graham Bell, the desk phone is becoming just one solution, out of many devices, used for the modern...
The emergence of cloud computing and Big Data warrants a greater role for the PMO to successfully manage enterprise transformation driven by these powerful trends. As the adoption of cloud-based services continues to grow, a governance model is needed to orchestrate enterprise cloud implementations and harness the power of Big Data analytics. In his session at 15th Cloud Expo, Mahesh Singh, President of BigData, Inc., to discuss how the Enterprise PMO takes center stage not only in developing th...
The Open Group and BriefingsDirect recently assembled a distinguished panel at The Open Group Boston Conference 2014 to explore the practical implications and limits of the Internet of Things. This so-called Internet of Things means more data, more cloud connectivity and management, and an additional tier of “things” that are going to be part of the mobile edge -- and extending that mobile edge ever deeper into even our own bodies. Yet the Internet of Things is more than the “things” – it me...
Software is eating the world. Companies that were not previously in the technology space now find themselves competing with Google and Amazon on speed of innovation. As the innovation cycle accelerates, companies must embrace rapid and constant change to both applications and their infrastructure, and find a way to deliver speed and agility of development without sacrificing reliability or efficiency of operations. In her keynote DevOps Summit, Victoria Livschitz, CEO of Qubell, will discuss ho...
In today's application economy, enterprise organizations realize that it's their applications that are the heart and soul of their business. If their application users have a bad experience, their revenue and reputation are at stake. In his session at 15th Cloud Expo, Anand Akela, Senior Director of Product Marketing for Application Performance Management at CA Technologies, will discuss how a user-centric Application Performance Management solution can help inspire your users with every appli...
Enthusiasm for the Internet of Things has reached an all-time high. In 2013 alone, venture capitalists spent more than $1 billion dollars investing in the IoT space. With “smart” appliances and devices, IoT covers wearable smart devices, cloud services to hardware companies. Nest, a Google company, detects temperatures inside homes and automatically adjusts it by tracking its user’s habit. These technologies are quickly developing and with it come challenges such as bridging infrastructure gaps,...
Predicted by Gartner to add $1.9 trillion to the global economy by 2020, the Internet of Everything (IoE) is based on the idea that devices, systems and services will connect in simple, transparent ways, enabling seamless interactions among devices across brands and sectors. As this vision unfolds, it is clear that no single company can accomplish the level of interoperability required to support the horizontal aspects of the IoE. The AllSeen Alliance, announced in December 2013, was formed wi...
Goodness there is a lot of talk about cloud computing. This ‘talk and chatter’ is part of the problem, i.e., we look at it, we prod it and we might even test it out – but do we get down to practical implementation, deployment and (if you happen to be a fan of the term) actual cloud ‘rollout’ today? Cloud offers the promise of a new era they say – and a new style of IT at that. But this again is the problem and we know that cloud can only deliver on the promises it makes if it is part of a well...
There’s Big Data, then there’s really Big Data from the Internet of Things. IoT is evolving to include many data possibilities like new types of event, log and network data. The volumes are enormous, generating tens of billions of logs per day, which raise data challenges. Early IoT deployments are relying heavily on both the cloud and managed service providers to navigate these challenges. In her session at 6th Big Data Expo®, Hannah Smalltree, Director at Treasure Data, to discuss how IoT, B...