Refuel Updates (Nov '24)

👋 We’re back with another edition of Refuel updates!

Here’s what’s new with Refuel from this past month:

1. Snowflake Integration

We’ve launched a Snowflake integration that makes it easy to structure, clean, and standardize your data from directly from inside of your data warehouse, with minimal engineering overhead.
 
Explore use cases, such as enriching financial transactions, call transcript extraction, address normalization, PDF extractions, and performing entity resolution - all from messy data in S3 and Snowflake.

2. Structured Outputs

You can now generate outputs in Refuel in accordance to a JSON schema. This structure can help ensure that the response is more organized, interpretable, and easier to use in applications.

3. Large Taxonomy Support

We’ve been running into more and more instances of customers needed to classify between large taxonomies.

For example:

• Healthcare companies need to classify diseases between 70,000 ICD codes

• Supply chain companies need to classify goods between thousands of HS codes

• HR companies need to classify and map skills between tens of thousands of possible options

Traditional ML models desperately fall short and even modern LLM’s struggle to fit large taxonomies within their context windows - retrieval now ends up being a critical part of the solution.

We’ve introduced large taxonomy support within Refuel to make it easier to run classification tasks amongst a vast taxonomy.

Simply upload the taxonomy as a CSV, specify the classification instructions, and we’ll take care of the rest.

Bonus:

• We wrote about evaluating and choosing the right SLM for your use case

• pip install --upgrade refuel to get the latest features and usability improvements.

• Latest docs available here

Let us know if you have questions or feature suggestions – have a great Thanksgiving and see you next month!