Between the 1970s and late 1990s, the sub-prime market was very small, and logically so. After all, who wants to make a loan, the loanee isn't likely to pay back? I'm broke and have no job, can you give me money? Err.. no.
What the CRA did was push banks to make the loans. When banks start make such loans, it legitimizes the action to other banks. Hey, Bear Stearns is make tons of sub-prime loans, and they are still in business! (Bear Stearns btw, was the first bank to take up Fannie Mae on marketing sub-prime loans)
More over, the cost of litigation with ACORN and other liberal nut case organizations, is very high. To make a few million sub-prime loans to keep the idiots off the law suit fence, is worth it in many cases.
Finely, Fannie Mae and Freddie Mac were both forced by HUD to purchase more sub-prime loans. This is a big big deal. If Bank of America is going to be sued if they don't offer sub-prime loans, but if they do, can sell them off to Fannie Mae, that's a darn good deal.
Both of these situations, validated the sub-prime market, and caused the risky loans to seem safe.
This isn't to excuse the banks from growing a spine, standing up to the government, and realizing the dangers of their actions. But we are trying to find the origin of the problem. If the CRA had not pushed banks to make these loans, they never would have happened. The market would never have been validated. The companies would never have made risky loans with such a low chance of being repaid.