🪴 Project Activity

👩‍🚀 Introduction

A personal research and development (R&D) lab that facilitates the sharing of knowledge.

🚀 Architecture

The diagram illustrates the repository's architecture, which is considered overly complex. It is essential to thoroughly understand the tradeoffs associated with before onboarding any technology into your project.

(The diagram here may take a moment to load. Please wait patiently.)

📦 Setup

make kubernetes-set-up

make kubernetes-clean

🧱 Tech Stack

Web

• React - Web framework
• Redux - State container
• React Query - Hooks for fetching, caching and updating asynchronous data
• redux-observable - Side effects
• RxJS - Asynchronous programming with observable streams
• graphql-tag - GraphQL query parsing
• Bulma - CSS framework
• PurgeCSS - Unused CSS removing
• Jest - Unit testing, snapshot testing
• React Testing Library - React component testing
• Storybook - Visual testing
  • Chromatic - Storybook reviewing
• rxjs/testing - Marble testing
• Cypress - End-to-end testing
• Lighthouse CI - Performance, accessibility, search engine optimization (SEO), progressive web app (PWA) analysis
• Sentry - Error tracking
• Report URI - Security reporting
• Google Tag Manager - Tag management
• Google Analytics - Web analytics
• FullStory - Experience analytics, session replay, heatmaps
• Namecheap - Domain
• Cloudflare - DNS, DDoS protection, CDN
• HTTP/3 Check - HTTP/3 checking
• hstspreload.org - HSTS checking
• Mozilla Observatory - Security monitoring
• UptimeRobot - Uptime monitoring

Mobile - iOS

• SwiftUI - UI framework
• XCTest - Unit testing, performance testing
• Slather - Code coverage reports generating

Mobile - Android

• AndroidX - Android Jetpack
• JUnit - Unit testing, instrumented testing

Mobile - React Native

• Expo - Universal native apps making platform
• React Native - Mobile application framework
• UI Kitten - UI library
• React Native Testing Library - React Native component testing

Desktop - Qt

• Qt Quick - Cross-platform application development framework
• QML - Qt modeling language

API Server - Go

• Gin - Web framework
• gRPC - Remote procedure call (RPC) framework
• graphql-go - GraphQL
• jwt-go - JSON Web Token (JWT)
• gin-contrib/cors - Cross-Origin Resource Sharing (CORS)
• opa - Open Policy Agent
• dgo - Dgraph client
• minio-go - MinIO client
• go-redis - Redis client
• pgx - PostgreSQL driver
• Resty - HTTP client
• Squirrel - SQL query builder
• apm-agent-go - Application performance monitoring (APM) agent
• OpenTelemetry Go - OpenTelemetry
• Prometheus Go - Prometheus
• Testify - Unit testing
• GoDotEnv - Environment variables loading
• jsonparser - JSON parser
• zerolog - Logging

API Server - Python

• Flask - Web framework
• Flask-CORS - Cross-Origin Resource Sharing (CORS)
• Gunicorn - Python web server gateway interface (WSGI) HTTP server
• asyncpg - PostgreSQL client
• Tenacity - General-purpose retrying library
• pytest - Unit testing
• pydantic - Data validation
• HTTPX - HTTP client
• pypdf - PDF library
• Poetry - Python package management
  • Poe the Poet - Poetry task runner

API Server - Node.js

• Express - Web framework
• GraphQL.js, express-graphql - GraphQL
  • graphql-ws, graphql-subscriptions - GraphQL subscriptions
  • graphql-upload - GraphQL upload
  • graphql-shield - GraphQL permissions
  • graphql-depth-limit - GraphQL depth limit
  • graphql-query-complexity - GraphQL query complexity analysis
• DataLoader - Batching and caching
• Knex.js - SQL query builder
• node-postgres - PostgreSQL client
• ioredis - Redis client
• rate-limiter-flexible - Rate limiting
• expressjs/cors - Cross-Origin Resource Sharing (CORS)
• csurf - CSRF protection
• jsonwebtoken, express-jwt - JSON Web Token (JWT)
• bcrypt - Password hashing
• axios - HTTP client
• Helmet - HTTP header Content-Security-Policy, Referrer-Policy, Strict-Transport-Security, X-Content-Type-Options, X-DNS-Prefetch-Control, X-Download-Options, X-Frame-Options, X-Permitted-Cross-Domain-Policies, X-XSS-Protection
• Report To - HTTP header Report-To
• Network Error Logging - HTTP header NEL
• express-request-id - HTTP header X-Request-ID
• response-time - HTTP header X-Response-Time
• connect-timeout - Request timeout
• Terminus - Health check and graceful shutdown
• pino - Logging
• dotenv-flow - Environment variables loading
• Stryker - Mutation testing
• SuperTest - HTTP testing
• autocannon - HTTP benchmarking
• Clinic.js - Performance profiling
• Node.js - JavaScript runtime
• npm - JavaScript package management

Cloud Native

• Sealed Secrets - Kubernetes secret encrypting
• ExternalDNS - Kubernetes services and Ingresses exposing
• cert-manager - Kubernetes X.509 certificate management
• Hasura - GraphQL Engine
  • hasura-metric-adapter - Hasura GraphQL Engine metric adapter
• Linkerd - Service mesh
• Caddy - Web server, reverse proxy, load balancer
• Traefik - Web server, reverse proxy, load balancer
• nginx - Web server, reverse proxy, load balancer
• Metrics Server - Kubernetes metrics
• Elastic APM - Application performance monitoring
• OpenTelemetry - Observability framework
• Jaeger - Distributed tracing system
• Netdata - Distributed monitoring platform
• Telegraf - Plugin-driven server agent
• Thanos - Highly available Prometheus setup with long-term storage capabilities
• Pixie - Observability tool for Kubernetes applications
• Docker - Container
• Skaffold - Continuous development for Kubernetes applications
• Multipass - VM management
• Locust - Load testing
• Cloudflare Tunnel - Tunneling
• Vertical Pod Autoscaler - Kubernetes vertical pod autoscaler
• K3s - Lightweight Kubernetes
• containerd - Container runtime
• Kubernetes - Container-orchestration system

Data

Database, Data Warehouse, Data Lakehouse

• Trino - Distributed SQL query engine
• PostgreSQL - Object-relational database
  • Postgres Operator - PostgreSQL high-availability (HA) template
  • Postgres Operator - PostgreSQL cluster provisioning
  • pgAdmin - PostgreSQL management tool
• MySQL - Relational database
  • MariaDB - Fork of MySQL
• Hydra - Column-oriented SQL database
• ClickHouse - Column-oriented SQL database
• YugabyteDB - Distributed SQL database
• TimescaleDB - Time-series SQL database
• InfluxDB - Time-series database
  • InfluxDB Enterprise - Distributed time-series database
• Prometheus - Time-series database
• Loki - Log aggregation system
• Apache Cassandra - Distributed wide-column NoSQL database
• Qdrant - Distributed vector database
• Chroma - Distributed vector database
• Dgraph - Distributed graph database
• Elasticsearch - Distributed document-oriented search engine
  • Kibana - Elasticsearch visualization
• Redis - Distributed in-memory key–value database
  • KeyDB - Multithreaded fork of Redis
• MinIO - Object storage
• Apache ZooKeeper - Distributed coordination system
• Apache Hadoop - Software utility collection
  • Apache Hadoop HDFS (Distributed File System) - Distributed file system
  • Apache Hadoop YARN (Yet Another Resource Negotiator) - Resource management and job scheduling framework
  • Apache Hadoop MapReduce - Data processing framework
• Apache Hive - Distributed data warehousing and SQL-like query language system built on top of Apache Hadoop
• Delta Lake - Data lakehouse
• Snowflake - Data warehouse
• golang-migrate/migrate - Database migrations

Data Ingestion

• Airbyte - Data integration
• Vector - Log collector
• Fluent Bit - Log collector

Data Orchestration

• Prefect - Orchestration platform
• Apache Airflow - Orchestration platform
• Temporal - Orchestration platform

Data Processing

• Apache Spark - Data processing framework
  • Spark ML - Spark machine learning
  • pyspark - Spark API library
  • Delight - Spark UI and history server
• Apache Sedona - Spatial data processing framework
• Apache Flink - Data processing framework
  • flink-streaming-java - Flink
  • flink-connector-twitter - Flink Twitter connector
  • flink-connector-jdbc - Flink JDBC Connector
  • flink-connector-redis - Flink Redis connector
• Apache Kafka - Distributed event streaming platform
  • Client
    • librdkafka - Kafka C/C++ client
      • libserdes - AVRO serialization and deserialization
    • confluent-kafka - Kafka Python client
  • Schema registry
    • Confluent Schema Registry - Schema Registry