How to effectively use BigQuery, avoid common mistakes, and execute sophisticated queries against large datasets Google BigQuery Analytics is the perfect guide for business and data analysts who want the latest tips on running complex queries and writing code to communicate with the BigQuery API. The book uses real-world examples to demonstrate current best practices and techniques, and also explains and demonstrates streaming ingestion, transformation via Hadoop in Google Compute engine, AppEngine datastore integration, and using GViz with Tableau to generate charts of query results. In addition to the mechanics of BigQuery, the book also covers the architecture of the underlying Dremel query engine, providing a thorough understanding that leads to better query results. Features a companion website that includes all code and data sets from the book Uses real-world examples to explain everything analysts need to know to effectively use BigQuery Includes web application examples coded in Python
- Introduction
- Part I BigQuery Fundamentals
- Chapter 1 The Story of Big Data at Google
- Big Data Stack 1.0
- Big Data Stack 2.0 (and Beyond)
- Open Source Stack
- Google Cloud Platform
- Cloud Processing
- Cloud Storage
- Cloud Analytics
- Problem Statement
- What Is Big Data?
- Why Big Data?
- Why Do You Need New Ways to Process Big Data?
- How Can You Read a Terabyte in a Second?
- What about MapReduce?
- How Can You Ask Questions of Your Big Data and Quickly Get Answers?
- Summary
- Chapter 2 BigQuery Fundamentals
- What Is BigQuery?
- SQL Queries over Big Data
- Cloud Storage System
- Distributed Cloud Computing
- Analytics as a Service (AaaS?)
- What BigQuery Isn't
- BigQuery Technology Stack
- Google Cloud Platform
- BigQuery Service History
- BigQuery Sensors Application
- Sensor Client Android App
- BigQuery Sensors AppEngine App
- Running Ad-Hoc Queries
- Summary
- Chapter 3 Getting Started with BigQuery
- Creating a Project
- Google APIs Console
- Free Tier Limitations and Billing
- Running Your First Query
- Loading Data
- Using the Command-Line Client
- Install and Setup
- Using the Client
- Service Account Access
- Setting Up Google Cloud Storage
- Development Environment
- Python Libraries
- Java Libraries
- Additional Tools
- Summary
- Chapter 4 Understanding the BigQuery Object Model
- Projects
- Project Names
- Project Billing
- Project Access Control
- Projects and AppEngine
- BigQuery Data
- Naming in BigQuery
- Schemas
- Tables
- Datasets
- Jobs
- Job Components
- BigQuery Billing and Quotas
- Storage Costs
- Processing Costs
- Query RPCs
- TableData.insertAll() RPCs
- Data Model for End-to-End Application
- Project
- Datasets
- Tables
- Summary
- Part II Basic BigQuery
- Chapter 5 Talking to the BigQuery API
- Introduction to Google APIs
- Authenticating API Access
- RESTful Web Services for the SOAP-Less Masses
- Discovering Google APIs
- Common Operations
- BigQuery REST Collections
- Projects
- Datasets
- Tables
- TableData
- Jobs
- BigQuery API Tour
- Error Handling in BigQuery
- Summary
- Chapter 6 Loading Data
- Bulk Loads
- Moving Bytes
- Destination Table
- Data Formats
- Errors
- Limits and Quotas
- Streaming Inserts
- Summary
- Chapter 7 Running Queries
- BigQuery Query API
- Query API Methods
- Query API Features
- Query Billing and Quotas
- BigQuery Query Language
- BigQuery SQL in Five Queries
- Differences from Standard SQL
- Summary
- Chapter 8 Putting It Together
- A Quick Tour
- Mobile Client
- Monitoring Service
- Log Collection Service
- Log Trampoline
- Dashboard
- Data Caching
- Data Transformation
- Web Client
- Summary
- Part III Advanced BigQuery
- Chapter 9 Understanding Query Execution
- Background
- Storage Architecture
- Colossus File System (CFS)
- ColumnIO
- Durability and Availability
- Query Processing
- Dremel Serving Trees
- Architecture Comparisons
- Relational Databases
- MapReduce
- Summary
- Chapter 10 Advanced Queries
- Advanced SQL
- Subqueries
- Combining Tables: Implicit UNION and JOIN
- Analytic and Windowing Functions
- BigQuery SQL Extensions
- The EACH Keyword
- Data Sampling
- Repeated Fields
- Query Errors
- Result Too Large
- Resources Exceeded
- Recipes
- Pivot
- Cohort Analysis
- Parallel Lists
- Exact Count Distinct
- Trailing Averages
- Finding Concurrency
- Summary
- Chapter 11 Managing Data Stored in BigQuery
- Query Caching
- Result Caching
- Table Snapshots
- AppEngine Datastore Integration
- Simple Kind
- Mixing Types
- Final Thoughts
- Metatables and Table Sharding
- Time Travel
- Selecting Tables
- Summary
- Part IV BigQuery Applications
- Chapter 12 External Data Processing
- Getting Data Out of BigQuery
- Extract Jobs
- TableData.list()
- AppEngine MapReduce
- Sequential Solution
- Basic AppEngine MapReduce
- BigQuery Integration
- Using BigQuery with Hadoop
- Querying BigQuery from a Spreadsheet
- BigQuery Queries in Google Spreadsheets (Apps Script)
- BigQuery Queries in Microsoft Excel
- Summary
- Chapter 13 Using BigQuery from Third-Party Tools
- BigQuery Adapters
- Simba ODBC Connector
- JDBC Connection Options
- Client-Side Encryption with Encrypted BigQuery
- Scientifi c Data Processing Tools in BigQuery
- BigQuery from R
- Python Pandas and BigQuery
- Visualizing Data in BigQuery
- Visualizing Your BigQuery Data with Tableau
- Visualizing Your BigQuery Data with BIME
- Other Data Visualization Options
- Summary
- Chapter 14 Querying Google Data Sources
- Google Analytics
- Setting Up BigQuery Access
- Table Schema
- Querying the Tables
- Google AdSense
- Table Structure
- Leveraging BigQuery
- Google Cloud Storage
- Summary
- Index
新书报道