Overview

  • Overarching Research Question: Analyze if the ongoing renovation project improved the reliability and performance of the MBTA system. Are there early indicators of significant improvement to the usage and performance of the MBTA system?

  • Goal: Gather and compile datasets relevant to our research question and its sub-questions. We identified reputable data sources and defined our collection methods by choosing an dataset accessed through an API, while finding datasets that have features which fit into requirements of future binary, multivariate, and regression analysis.

  • Motivation:

    • Centralize Data Access: This is a huge project and a tough question to answer! Consolidating datasets to one source not only makes our analysis easier, but also helps Jen and I collaborate.
    • Contextual Insights: We want to collect as much datasets to inform our question. The more, the better. We want to gain a holistic understanding of MBTA from its performance to public perceptions
  • Objectives:

    • Identify and Gather Relevant and Reliable Datasts
    • Ensure Data Coverage (2020-2024)
    • Document data sources
    • Validate data accessibility so recreation is possible
  • Dataset Info:

    Data source 1

    • Datasource: MBTA Bluebook Open Data Portal
    • Type: Tabular (.csv)
    • Structure: 2,736,828 rows, 28 features (combined)
    • Years: 2020-2024
    • Purpose:
      • Analyze causes and effects of service disruptions.
      • Identify routes and stations prone to frequent alerts.
      • Measure severity and frequency of alerts over time.

    Data source 2

    • Datasource: Serpapi: Google News Feed
    • Type: Tabular (.json), API
    • Structure: 100 entries
    • Years: 2024
    • Purpose:
      • Understand current public perception through text analysis (sentimen analysis, topic modeling, keyword extraction)
      • Technical use of APIs

    Data source 3

    • Datasource: MBTA Bluebook Open Data Portal
    • Type: Tabular (.csv)
    • Structure: 3682 rows, 7 features
    • Years: 2014-2024
    • Purpose:
      • Evlauate long-term trends in service reliability
      • Identify improvements or challenges in accessibility services.

    Data source 4

    • Datasource: MBTA Bluebook Open Data Portal
    • Type: Tabular (.json)
    • Structure: 7185 rows, 10 features
    • Years: 2020-2024
    • Purpose:
      • Find differences between predicted and actual service performance
      • Evalute prediction accuracy trends

    Data source 5

    • Datasource: Financial Statements
    • Type: PDF (.json)
    • Structure: 5 pdfs (transformed to 8 rows, 5 features during cleaning)
    • Years: 2019-2024
    • Purpose:
      • Understand macro-level trends in financial trends and whether it correlates with system performance (i.e. reliability dataset)
      • examine important money values such as the net positions, expenses, revenues, and debts throughout the years from 2019 to 2023