💰 FinDAP:
Demystifying Domain-adaptive Post-training for Financial LLMs

EMNLP 2025 🏅 Oral Presentation (Top 50% of accepted papers, ARR best paper nomination)
Zixuan Ke Zixuan Ke Yifei Ming Yifei Ming Xuan-Phi Nguyen Xuan-Phi Nguyen Caiming Xiong Caiming Xiong Shafiq Joty Shafiq Joty
Salesforce AI Research
[📄 Paper] [🤗 Llama-Fin-8b] [🤗 FinTrain] [🤗 FinEval] [💻 Code]

Research Questions

Given a strong general-purpose LLM (e.g., Llama3-8b-inst), how can we effectively adapt it to a target domain (e.g., finance) through post-training?

Key Questions We Address:

  • What criteria are desirable for successful adaptation?
  • What are the most effective training recipes with respect to data and model?
  • How do different post-training stages contribute to domain expertise?

Abstract

Domain-adaptive post-training of large language models (LLMs) has emerged as a promising approach for specialized domains such as medicine and finance. However, significant challenges remain in identifying optimal adaptation criteria and training strategies across varying data and model configurations.

To address these challenges, we introduce FinDAP, a systematic and fine-grained investigation into domain-adaptive post-training of LLMs for the finance domain. Our approach consists of four key components:

FinCap defines the core capabilities required for the target domain

FinRec provides an effective training recipe that jointly optimizes continual pre-training and instruction-following, along with a novel preference data distillation method leveraging process signals from a generative reward model

FinTrain offers a curated set of training datasets supporting FinRec

FinEval delivers a comprehensive evaluation suite aligned with FinCap

The resulting model, Llama-Fin, achieves state-of-the-art performance across a wide range of financial tasks. Our analysis also highlights how each post-training stage contributes to distinct capabilities, uncovering specific challenges and effective solutions, providing valuable insights for domain adaptation of LLMs.

🎯 Key Contributions

📊 Comprehensive guidance

for finance-specific post-training, including identification of capabilities, evaluation, data and model recipe design.

🔬 Systematic exploration

on each stage of post-training, with an emphasis on the goals, challenges and effective approaches.

📈 Novel preference alignment approach

that constructs preference data using on-policy trajectories guided by outcome and process signals.

💡 New State-of-the-art financial LLM

(Llama-Fin) at the 8b parameter scale based on the above.

🏗️ FinDAP Framework Overview

FinDAP Framework Overview
Figure 1: FinDAP framework overview - Hover over components to explore details
⬇️ Interactive Component Details ⬇️

🔍 Explore Each Component

Click below to dive deeper into the methodologies and technical details of each framework component

🎯 FinCap: Financial Capabilities Framework

+

Systematic identification and definition of core financial LLM capabilities

🔧 FinRec: Training Recipe & Methodology

+

Advanced training strategies and novel preference alignment techniques

📚 FinTrain: Curated Training Data

+

Systematically curated datasets for optimal financial domain adaptation

📊 FinEval: Comprehensive Evaluation Suite

+

Multi-dimensional evaluation framework for financial LLM assessment

🚀 Explore Our Work

Dive deep into our systematic approach to financial LLM adaptation

📝 Citation

@misc{ke2025demystifyingdomainadaptiveposttrainingfinancial,
      title={Demystifying Domain-adaptive Post-training for Financial LLMs}, 
      author={Zixuan Ke and Yifei Ming and Xuan-Phi Nguyen and Caiming Xiong and Shafiq Joty},
      year={2025},
      eprint={2501.04961},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2501.04961}, 
}