brunoamaral/gregory-ai

Major Features

• "Latest Research by Category" for better content organization
• Enhanced Machine Learning with BERT, LGBM, and LSTM models
• Clinical Trials and Article cross-references
• BioRxiv and MedRxiv repository support
• Completely redesigned email templates
• API improvements including author full_name property

Improvements

• Better admin interface with health indicators and batch actions
• Improved error handling, especially for Unpaywall API
• Consolidated database migrations
• Extensive test coverage added across the application

Bug Fixes

• Fixed management command verbosity issues
• Resolved ORCID query problems
• Fixed various UI and template rendering issues
• Improved email content handling

Full release notes

Major Features

1. "Latest Research by Category" Feature

• Added a ManyToManyField to TeamCategory for latest_research_categories in Lists model
• Updated admin interface and forms for better category selection
• Created utility functions to fetch latest articles by team category (limited to 20 per category)
• Updated email templates to group and display research by category
• Added comprehensive tests for the feature
• Documentation: Latest Research Categories

2. Machine Learning Improvements

• Complete overhaul of ML subsystem with new architecture and algorithms
• Added new ML commands: train_models and predict_articles
• Added algorithm tracking to ML predictions
• Implemented BERT, LGBM, and LSTM models
• Added ML predictions to API responses
• Improved ML admin interface with better visualization
• Added extensive test coverage for ML features
• Documentation: Machine Learning, Algorithm Tracking

3. Clinical Trials and Article References

• Added relationship between clinical trials and articles
• Added trial references to article API responses and vice versa
• Created management command to detect trial references
• Added tests for trial reference detection

4. Additional Repository Support

• Added support for BioRxiv and MedRxiv RSS feeds
• Improved handling of abstracts and summaries
• Added keyword filtering capabilities for sources
• Documentation: BioRxiv/MedRxiv Keyword Filtering

5. Email Template Improvements

• Completely restructured email templates with components
• Added preview and testing system for email templates
• Improved styling and consistency across all email types
• Better organization of content in weekly summary emails

6. API Enhancements

• Added full_name property to author serializers
• Improved API documentation
• Added new endpoints for categories and teams
• Fixed URL generation in API responses

Improvements

1. Admin Interface

• Added new admin page for article review status
• Improved author management in admin interface
• Added health indicators to sources
• Enhanced ML prediction display in admin
• Added batch actions to admin pages

2. Error Handling

• Improved Unpaywall API error handling
• Better JSON decoding error handling
• More robust error handling throughout the application
• Documentation: Unpaywall Error Handling Fix

3. Database and Migrations

• Simplified and consolidated migrations
• Fixed team foreign key constraints
• Updated database models for better relationships
• Documentation: Migration Fix

4. Testing

• Added extensive test coverage across the application
• Tests for management commands
• Tests for API serializers and endpoints
• Tests for ML components
• Tests for subscription features

Bug Fixes

1. Fixed verbosity argument conflict in management commands
2. Fixed ORCID server query issues
3. Fixed 500 server error when updating authors' ORCID
4. Fixed pipeline execution errors
5. Fixed template rendering issues
6. Fixed email content limitations
7. Fixed database constraints issues
8. Resolved null value handling in models
9. Fixed dark mode styling issues

Technical Debt Reduction

1. Removed deprecated code and migrations
2. Improved code organization
3. Enhanced documentation
4. Better error logging
5. Performance improvements for database queries
6. Standardized API response formats

Development Tools

1. Added Codex workflow for issue automation
2. Improved development environment setup
3. Updated requirements.txt for better dependency management
4. Added debugging tools and utilities

This release represents a significant improvement in functionality, reliability, and code quality. The machine learning capabilities have been greatly enhanced, the email system has been modernized, and many bugs have been fixed.

What's Changed

• adds a new fresh migration by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/502
• Implement new ML models for predicting article relevance by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/503
• Adds new email templates made by Claude by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/505
• adds better error message by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/507
• 508 number of trials in title does not match total trials listed by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/509
• fixes the lack of ML predictions by using the new db relationship by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/510
• Improvements to Organisations and teams by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/511
• fixes #512 by repairing db constraints by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/513
• Email templates customization and clean up by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/514
• 413 pubmed articles return a truncated abstract by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/515
• adds support for BioRxiv RSS by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/516
• Fix docs for ML training and team endpoints by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/517
• Add Codex workflow to auto assign labeled issues by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/518
• Handle multi-list subscription by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/520
• implements a template filter to remove tags from strings by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/524
• Use current Site domain in redirects by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/525
• Filter admin relevancies by auto_predict by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/526
• adds a relationship between clinical trials and articles by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/527
• adds the relationship between articles and clinical trials to the API… by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/528
• Add Django command tests by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/529
• fixes query to the orcid server by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/531
• improves the admin page for authors by listing the articles, allowing sorting and filtering by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/532
• adds a verbosity flag to feedreader commands by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/533
• adds health indicators to sources admin page by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/534
• add latest research list based on team categories to the weekly summary by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/535
• adds error handling to the unpaywall request and associated test by @brunoamaral in https://github.com/brunoamaral/gregory-ai/pull/537

Full Changelog: https://github.com/brunoamaral/gregory-ai/compare/v22...v23