I love banana bread! Unfortunately, it is always loaded in fat when you buy it or follow traditional recipes. There are tons of low fat recipe options out there but the always take out the walnut. Removing them will take away from those "scary numbers" we all watch; however, walnuts are such a rich source of nutrition that they are worth keeping.
Side Note: Many people use "Splenda" as a sugar substitute, if you get an icky aftertaste try Stevia, like "Truvia". I personally prefer "Sun Crystal", a mix of natural sugar cane and stevia. Ultimately, it's your choice.