Implemented an RDBMS in Java, with Visitor Design Pattern and composite data structures, supporting nearly full SQL syntax, including implicit join, expression updates, and arbitrary expression evaluation for WHERE conditions.

Implemented cost-based and rule-based query optimization, integrity constraints and achieved sub-1-second responsiveness manipulating up to one-million-record tables, which is close to commercial grade.

Scraped and collected feature data for over 500K songs and artists from sources including Spotify, Wikipedia and allmusic.com, larger than any public dataset, stored in MongoDB NoSQL database. Worked on missing value imputation, outliers identification and duplication removal with statistical methods using Pandas, Numpy and Scikit-learn.

Analysed influence and trends caused by popular artists with clustering, regression, ANOVA, classification, and network analysis using Scikit-learn, Scipy and NetworkX. Utilized Matplotlib and Plotly for visualization.

Built a high-quality dataset to describe behaviors of taxi drivers in Qingdao using multiple dimensions from hundreds of Gigabytes of raw GPS data with a distributed system in the workflow of MongoDB and Spark; extracted multiple features like the empty rate, work time, and profit from the spatial–temporal data.

Designed a brand-new multi-input RNN model with human-related features, environment-related features, and income data input simultaneously using PyTorch; the RMSE for predicting drivers’ income improved by 8.3% using the dataset as compared to LSTM.