I just skimmed through the entire wiki article about what happened to Frank's family, and I'm even more confused. What I got was that they were collateral damage from someone (police? military? someone else? idk) taking down someone who was involved with whatever Frank's team had been involved with in Afghanistan. Billy knew what was going down that day and had been hanging out with the Castle family that day and didn't do anything to get them out of there. So by doing nothing, he's responsible.