Welcome!

Big Data Journal Authors: Carmen Gonzalez, Liz McMillan, Victoria Livschitz, Elizabeth White, Pat Romanski

Blog Feed Post

Well Engineered use of AWS by Recovery and Transparency Board (RATB)

By

After speaking with Shawn Kingsberry in preparations for our 4 April Government Big Data Forum I realized their use of Amazon Web Services (AWS) may be of very high interest to our readers and went about looking for more info online to see what was publicly available. I was ecstatic to see a well written use case for much of it is on the AWS website. That write-up includes a nice graphic that is helpful to understanding how things were done.

Since this is provided by AWS as a way of articulating their special contributions it does not go into the many other services and components required to make this work. But many of those components are probably modular and exchangable with other capabilities. So this overview is probably a great way to get a baseline on what the RATB architecture is.

With that, the following is from: http://aws.amazon.com/solutions/case-studies/ratb/

AWS Case Study: Recovery.gov and AWS Bring Transparency to the Cloud

The Recovery Accountability and Transparency Board (RATB) was established when Congress passed the American Recovery and Reinvestment Act (ARRA) in February, 2009. To ensure against waste, fraud, and abuse, the RATB was tasked with developing a Website which met the following goals:

  • Provide easily accessible information to the public on Recovery spending and results
  • Promote official data in public debate
  • Provide fair and open access to Recovery opportunities
  • Enable public accountability for Recovery spending
  • Promote an understanding of the local impact of Recovery spending

The resulting Website is Recovery.gov.

The RATB originally intended to use Amazon Web Services (AWS) only for development, testing, and as failover, but, says Jim Warren, RATB Chief Information Officer, “When AWS outperformed our on-premises solution at a fraction of the cost, the prime contractor Smartronix and its lead sub-contractor Synteractive, provided a compelling justification for the RATB to host Recovery.gov on AWS’s platform.”

According to Mr. Warren, Smartronix selected AWS because of the flexibility provided by AWS’s Infrastructure as a Service (IaaS) model; track record of providing infrastructure for large-scale commercial projects; focus on cost-effectiveness and a pay-as-you-go-model that allowed Smartronix to control costs; commitment to security and reliability; and its FISMA Low certification.

The RATB now uses the following AWS services: Amazon Elastic Compute Cloud (Amazon EC2), Amazon Simple Storage Service (Amazon S3), Amazon Elastic Block Storage (Amazon EBS), Elastic Load Balancing (ELB), and Amazon CloudWatch. The solution also combined multiple pieces of software.

The following diagrams illustrate their topology:

 

ratb arch diagram Well Engineered use of AWS by Recovery and Transparency Board (RATB)

Recovery Accountability and Transparency Board

Business Intelligence and Data Warehousing
The website uses Microsoft’s SharePoint as it content management system and all data is aggregated into a global dimensional data warehouse to facilitate time-based analysis and reporting. The solution leverages SAP BusinessObjects and Microsoft SQL Server for reporting services that show how and where the money is being spent. The BI tools enable ad hoc reporting and are instrumental in Data Quality and Data Integrity score-carding.

Advanced Geospatial Analysis and Mapping
The Geospatial tools, based on ESRI software, allow up to 5,000 concurrent users and enables them to go directly to go to their communities of interest at the state, zip, congressional district, or county level. Hundreds of thousands of addresses are geo-coded and aggregated to display total value for each area of interest. Thematic maps and multiple view selections were incorporated to help the user better visualize the data. These thematics include funding heat maps, unemployment heat maps, and diversity maps.

Mr. Warren notes that testing and development enclaves were procured and ready on Amazon EC2 within two days of the contract award. He says, “Our migration to the cloud took only 22 days from feasibility study to production.” The RATB has also enjoyed improved computer security, including greater protection against network attacks and real-time detection of system tampering. Mr. Warren says, “In essence, the security system of AWS’s platform has been added to our existing security systems. We now have a security posture consistent with that of a multi-billion dollar company.” Additional benefits include lower costs and ability to add capacity on demand. The RATB expects to save around $750K during their current budget cycle.

The success of Recovery.gov is being noticed outside of the RATB as well: Andre Romano of Newsweek wrote, “The current incarnation of Recovery.gov…is perhaps the clearest, richest interactive database ever produced by the American bureaucracy.” The site has been given the 2009 Merit award, the 2010 Gold Addy award for Website design, InformationWeek Government IT Innovator 2010 Award, an Award of Distinction during the 16th Annual Communicator Awards, and a second place Gold Screen Award from the National Association of Government Communicators. Recovery.gov is also an official Honoree for the Financial Services category in the 14th Annual Webby Awards.

To learn more see http://recovery.gov

 Well Engineered use of AWS by Recovery and Transparency Board (RATB)

Read the original blog entry...

More Stories By Bob Gourley

Bob Gourley, former CTO of the Defense Intelligence Agency (DIA), is Founder and CTO of Crucial Point LLC, a technology research and advisory firm providing fact based technology reviews in support of venture capital, private equity and emerging technology firms. He has extensive industry experience in intelligence and security and was awarded an intelligence community meritorious achievement award by AFCEA in 2008, and has also been recognized as an Infoworld Top 25 CTO and as one of the most fascinating communicators in Government IT by GovFresh.

@BigDataExpo Stories
The 3rd International Internet of @ThingsExpo, co-located with the 16th International Cloud Expo - to be held June 9-11, 2015, at the Javits Center in New York City, NY - announces that its Call for Papers is now open. The Internet of Things (IoT) is the biggest idea since the creation of the Worldwide Web more than 20 years ago.
The 4th International DevOps Summit, co-located with16th International Cloud Expo – being held June 9-11, 2015, at the Javits Center in New York City, NY – announces that its Call for Papers is now open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's large...
SAP is delivering break-through innovation combined with fantastic user experience powered by the market-leading in-memory technology, SAP HANA. In his General Session at 15th Cloud Expo, Thorsten Leiduck, VP ISVs & Digital Commerce, SAP, discussed how SAP and partners provide cloud and hybrid cloud solutions as well as real-time Big Data offerings that help companies of all sizes and industries run better. SAP launched an application challenge to award the most innovative SAP HANA and SAP HANA...
Cultural, regulatory, environmental, political and economic (CREPE) conditions over the past decade are creating cross-industry solution spaces that require processes and technologies from both the Internet of Things (IoT), and Data Management and Analytics (DMA). These solution spaces are evolving into Sensor Analytics Ecosystems (SAE) that represent significant new opportunities for organizations of all types. Public Utilities throughout the world, providing electricity, natural gas and water,...
The security devil is always in the details of the attack: the ones you've endured, the ones you prepare yourself to fend off, and the ones that, you fear, will catch you completely unaware and defenseless. The Internet of Things (IoT) is nothing if not an endless proliferation of details. It's the vision of a world in which continuous Internet connectivity and addressability is embedded into a growing range of human artifacts, into the natural world, and even into our smartphones, appliances, a...
The Internet of Things is tied together with a thin strand that is known as time. Coincidentally, at the core of nearly all data analytics is a timestamp. When working with time series data there are a few core principles that everyone should consider, especially across datasets where time is the common boundary. In his session at Internet of @ThingsExpo, Jim Scott, Director of Enterprise Strategy & Architecture at MapR Technologies, discussed single-value, geo-spatial, and log time series dat...
How do APIs and IoT relate? The answer is not as simple as merely adding an API on top of a dumb device, but rather about understanding the architectural patterns for implementing an IoT fabric. There are typically two or three trends: Exposing the device to a management framework Exposing that management framework to a business centric logic Exposing that business layer and data to end users. This last trend is the IoT stack, which involves a new shift in the separation of what stuff happe...
An entirely new security model is needed for the Internet of Things, or is it? Can we save some old and tested controls for this new and different environment? In his session at @ThingsExpo, New York's at the Javits Center, Davi Ottenheimer, EMC Senior Director of Trust, reviewed hands-on lessons with IoT devices and reveal a new risk balance you might not expect. Davi Ottenheimer, EMC Senior Director of Trust, has more than nineteen years' experience managing global security operations and asse...
The Internet of Things will greatly expand the opportunities for data collection and new business models driven off of that data. In her session at @ThingsExpo, Esmeralda Swartz, CMO of MetraTech, discussed how for this to be effective you not only need to have infrastructure and operational models capable of utilizing this new phenomenon, but increasingly service providers will need to convince a skeptical public to participate. Get ready to show them the money!
"SAP had made a big transition into the cloud as we believe it has significant value for our customers, drives innovation and is easy to consume. When you look at the SAP portfolio, SAP HANA is the underlying platform and it powers all of our platforms and all of our analytics," explained Thorsten Leiduck, VP ISVs & Digital Commerce at SAP, in this SYS-CON.tv interview at 15th Cloud Expo, held Nov 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
Enthusiasm for the Internet of Things has reached an all-time high. In 2013 alone, venture capitalists spent more than $1 billion dollars investing in the IoT space. With "smart" appliances and devices, IoT covers wearable smart devices, cloud services to hardware companies. Nest, a Google company, detects temperatures inside homes and automatically adjusts it by tracking its user's habit. These technologies are quickly developing and with it come challenges such as bridging infrastructure gaps,...
"Cloud consumption is something we envision at Solgenia. That is trying to let the cloud spread to the user as a consumption, as utility computing. We want to allow the people to just pay for what they use, not a subscription model," explained Ermanno Bonifazi, CEO & Founder of Solgenia, in this SYS-CON.tv interview at Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
Explosive growth in connected devices. Enormous amounts of data for collection and analysis. Critical use of data for split-second decision making and actionable information. All three are factors in making the Internet of Things a reality. Yet, any one factor would have an IT organization pondering its infrastructure strategy. How should your organization enhance its IT framework to enable an Internet of Things implementation? In his session at Internet of @ThingsExpo, James Kirkland, Chief Ar...
SAP is delivering break-through innovation combined with fantastic user experience powered by the market-leading in-memory technology, SAP HANA. In his General Session at 15th Cloud Expo, Thorsten Leiduck, VP ISVs & Digital Commerce, SAP, discussed how SAP and partners provide cloud and hybrid cloud solutions as well as real-time Big Data offerings that help companies of all sizes and industries run better. SAP launched an application challenge to award the most innovative SAP HANA and SAP HANA...
Vichara Technologies in Hoboken, New Jersey is expanding its capabilities in big data from origins on Wall Street into other areas, and thereby demonstrating the growing marketplace for advanced big-data analytics services. The next BriefingsDirect deep-dive big data benefits case study interview explores how Vichara Technologies in Hoboken, New Jersey is expanding its capabilities in big data from origins on Wall Street into other areas, and thereby demonstrating the growing marketplace for ad...
The definition of IoT is not new, in fact it’s been around for over a decade. What has changed is the public's awareness that the technology we use on a daily basis has caught up on the vision of an always on, always connected world. If you look into the details of what comprises the IoT, you’ll see that it includes everything from cloud computing, Big Data analytics, “Things,” Web communication, applications, network, storage, etc. It is essentially including everything connected online from ha...
Scene scenario: 10 am in a boardroom somewhere, second round of coffees served, Danish and donuts untouched, a quiet hush settles. “Well you know what guys? (and, by the use of the term guys I mean to include both sexes here assembled) – the trouble that we have as a company is that we are, to put it bluntly, just a little analytics poor,” said the newly appointed Chief Analytics Officer. That we should consider a firm to be analytically deficient or poor is a profound comment on our modern ag...
Cloud Expo 2014 TV commercials will feature @ThingsExpo, which was launched in June, 2014 at New York City's Javits Center as the largest 'Internet of Things' event in the world.
SYS-CON Events announced today that Windstream, a leading provider of advanced network and cloud communications, has been named “Silver Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York, NY. Windstream (Nasdaq: WIN), a FORTUNE 500 and S&P 500 company, is a leading provider of advanced network communications, including cloud computing and managed services, to businesses nationwide. The company also offers broadband, p...
The major cloud platforms defy a simple, side-by-side analysis. Each of the major IaaS public-cloud platforms offers their own unique strengths and functionality. Options for on-site private cloud are diverse as well, and must be designed and deployed while taking existing legacy architecture and infrastructure into account. Then the reality is that most enterprises are embarking on a hybrid cloud strategy and programs. In this Power Panel at 15th Cloud Expo (http://www.CloudComputingExpo.com...