Data Learning Notes

Data Gathering from multiple sources / multiple datasets

Throughout my various projects, I’ve gained some valuable insights from projects, and I’d love to share them to help others. My goal is to enhance project outcomes by sharing these lessons.

When starting a data project, the initial phase often includes conducting thorough research to identify and gather one or multiple data sources from various channels and platforms.

A few things to take into consideration:

Note the URL of the initial source/s:. This will help to credit the source/s and reference if the features have non-descriptive names.
Bookmark or note related/notable data sources: This can help to enhance existing datasets and create additional features.
Create a data dictionary: This can be helpful so the team can reference and understand feature/column definitions.
Work to create/evaluate data standards early and often: This can help to ensure everyone has the same understanding when collecting and processing the data for final output (analytics, ML, reverse ETL).
Create a data process document: This is especially helpful when gathering information from different channels, websites, and formats. Don’t forget to stay on top of the data refresh schedule and keep an eye on the date of the last update.
For larger projects: — Develop a Framework for Data Management: outline the processes and procedures for managing data including data classification, retention, and disposal. This framework should be flexible to accommodate changing business needs. — Consider Data Governance: start small and learn, begin with a pilot project or a specific data domain.

Hopefully, by following these initial steps for your projects, you can save time and start developing robust data processes and workflows. It’s important to move towards a governance framework that supports business growth, reduces risks, and enhances decision-making capabilities.

Juanita

Data Learning Notes

Pages