BloombergGPT: How We Built a 50 Billion Parameter Financial Language Model

Home
My videos
BloombergGPT: How We Built a 50 Billion Parameter Financial Language Model

Summary

BloombergGPT is a 50 billion parameter language model trained on a balanced mix of general-purpose datasets and financial documents from the Bloomberg archives. Despite facing challenges during training, such as loss spikes and parameter drifts, BloombergGPT outperformed existing models on financial tasks while maintaining competitive performance on general benchmarks. The talk aims to provide insights into the challenges of building large language models and offer guidance for those considering embarking on their own language model journey.

Have questions about the video? Create a FREE account to ask Wiz AI

Sign up

7 days free trial