AI Deal Scoring for Private Lenders: Building a Claude Underwriter and a Back-Test Plan

A deal score is only worth trusting if it follows your written guidelines and you can show it was right on loans that have already closed. A private real estate lender asked us for both: an underwriting project built on Claude that scores incoming deals, and a way to test those scores against past outcomes.

The people on the call were credit, operations and sales leaders, and the work is relevant to mortgage loan officers and private lenders generally. Underwriting involves judgment, and judgment can vary between people. The goal here was a consistent first pass that flags exceptions, with the proof built in.

The situation

Deal underwriting needed a consistent, guideline-based scoring process. The lender has written loan guidelines and matrices, and the question was whether Claude could apply them the same way every time and flag anything that falls outside them.

The team also named a gap early. Permits and rehab risk were not yet part of the scoring. That is a real hole, and a team member raised it on the call, before the first test.

What was broken

In general, without a shared scoring method, two people reading the same deal can reach different conclusions, and neither can easily show why. A score that cannot be traced back to the guidelines is also hard to defend.

That is why one team leader flagged that the loan guidelines must be loaded into the project. If the tool does not have them, its scoring logic is incomplete, and it cannot flag exceptions.

What we built

We set up the project during the call and started the build of the scoring tool itself.

• A Claude project for deal underwriting, with the lender's context files, loan guidelines and matrices loaded in.
• An artifact build kicked off in the call, so the team has a scoring tool to finish and test instead of a document describing one.
• A test on a real deal before the next follow-up, so the team can see what needs to change.
• A review of the guidelines and risk checks inside the tool, assigned to a team leader, to find gaps such as the missing permit and rehab checks.
• A plan to batch-run about 60 closed loans and compare the initial scores to what actually happened: the appraisal, the sale price, the draw cadence and the timeline.
• A list of closed loans that have both the original submission data and the later results, which will seed the feedback loop that recalibrates the scoring.

What changed

No outcome has been reported. The tool had not yet been tested on a real deal when the call ended, and the back-test is a plan.

What changed is the method. It now has a home: a project that holds the guidelines, scores against them and can be checked against history. Two follow-ups were set: one for quick fixes after the real-deal test, and one on plugin readiness and what a borrower-facing version would cover.

How to back-test a scoring tool

This section is advice. Pick closed loans where you have both sides of the story: what you knew at submission and what happened afterward. Run them through the tool without telling it the results. Then compare each score with the real appraisal, sale price, draw pattern and timeline, and write down every loan where the score and the outcome disagree.

Treat the misses as the product. Each one points to a missing input, a guideline the tool misread or a risk that was never in the scoring. Keep a person responsible for the final credit decision, and treat the score as a second opinion that arrives earlier and applies the guidelines the same way each time.

Frequently asked questions

Can Claude underwrite a private real estate loan?

In this build, Claude scores deals against the lender's written guidelines and flags exceptions. We would treat the score as an input for the credit team rather than a decision, and the back-test exists to show how well it matches real outcomes.

Why load the loan guidelines into the project?

A team leader flagged that the guidelines have to be included so the tool follows them and flags exceptions. Without them the scoring logic is incomplete, because the tool would be guessing at the lender's rules instead of applying them.

What does back-testing on closed loans involve?

The plan is to run about 60 closed loans through the tool and compare the first scores with actual results, including appraisal, sale price, draw cadence and timeline. The loans need both the original submission data and the later outcome data.

What was missing from the scoring at the start?

Permits and rehab risk were not yet included. A team member pointed that out, and a team leader was assigned to validate the guidelines and risk checks in the tool and find other gaps before the next refinement session.

If your underwriting depends on who reads the file, book a 30-minute AI follow-up and reporting audit. We look at your guidelines and your closed-loan data and show you what a consistent first-pass score would need.