<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><span class="Apple-style-span" style="font-family: Calibri; font-size: 15px; ">Thanks Ken. For me this has been an incredibly eye-opening project. It can be hard for people to distinguish between ideas that merely look good on paper and ideas that are actually in widespread use. Once we’ve cleaned up the data and gotten approval from the organizations we canvassed, we think there’ll be plenty of ways to apply what we’ve learned. The project Pravir mentioned is one.</span></div><div><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt"> <br> Brian<br> <br></span></font></div><div><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt">[Ed. This was from <a href="mailto:Brian@fortify.com">Brian@fortify.com</a>, but was dropped by the Mailman server since it's set to ignore emails from non-subscribed addresses (spam...). Issue was resolved re Brian's email address.]<br> <br> On 12/17/08 11:48 AM, "Ken van Wyk" <<a href="Ken@KRvW.com">Ken@KRvW.com</a>> wrote:<br> <br> </span></font><blockquote type="cite"><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt">On Dec 16, 2008, at 1:25 PM, Gary McGraw wrote:<br> </span></font><blockquote type="cite"><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt">Using the software security framework introduced in October (A Software Security Framework: Working Towards a Realistic Maturity Model <<a href="http://www.informit.com/articles/article.aspx?p=1271382>">http://www.informit.com/articles/article.aspx?p=1271382></a>), we interviewed nine executives running top software security programs in order to gather real data from real programs. <br> </span></font></blockquote><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt"><br> Wow, this is great stuff. Kudos to Gary, Sammy, and Brian.<br> <br> I have a couple comments/observations on some of your conclusions:<br> <br> - You obviously wrote the top-10 list in C, since it went from 9 to 0. :-)<br> <br> - "N<i>ot only are there are no magic software security metrics, bad metrics actually hurt.</i>" This is an excellent point. I think it's also worth noting that it's important to carefully consider what metrics make sense for an organization _as early as possible_ in the life of their software security efforts. Trying to retro-engineer some metrics into a program after the fact is not a fun thing.<br> <br> - "<i>Secure-by-default frameworks can be very helpful, especially if they are presented as middleware classes (but watch out for an over focus on security "stuff").</i> " Yes yes yes! I've found significantly more "traction" to prescriptive guidance vs. a "don't do this" list of bad practices. Plus, it inherently supports a mindset of positive validation instead of negative. It's important to look for common mistakes, but if you really want your devs to follow, give them clear coding guidelines with annotated descriptions of how to follow them. Efforts like OWASP's ESAPI are indeed a great starting point here for plugging in things like strong positive input validation and such.<br> <br> - "<i>Web application firewalls are not in wide use, especially not as Web application firewalls</i>. " I can't say I'm much surprised by this one. Even with PCI-DSS driving people to WAFs (or do external independent code reviews), I just don't often see them often. But you go on to say, "But even these two didn't use them to block application attacks; they used them to monitor Web applications and gather data about attacks."--but you don't come back to this point. One serious benefit to WAFs can be enhancing the ability to do monitoring, especially of legacy apps. Adding one network choke point WAF can quickly add an app-level monitoring capability that few organizations considered when rolling the apps out in the first place.<br> <br> - "<i>Though software security often seems to fit an audit role rather naturally, many successful programs evangelize (and provide software security resources) rather than audit even in regulated industries</i>" This one too is very encouraging to see.<br> <br> - "<i>Architecture analysis is just as hard as we thought, and maybe harder.</i>" And this one is very discouraging. I've seen good results in doing architectural risk analyses, but the ones that produce useful results tend to be the more ad hoc ones -- and NOT the ones that follow rigorous processes.<br> <br> - "<i>All nine programs we talked to have in-house training curricula, and training is considered the most important software security practice in the two most mature software security initiatives we interviewed.</i> " That explains the quarter-million miles in my United account this year alone. :-) Ugh.<br> <br> - "<i>Though all of the organizations we talked to do some kind of penetration testing, the role of penetration testing in all nine practices is diminishing over time.</i> " Hallelujah!<br> <br> <br> <br> Cheers,<br> <br> Ken<br> <br> -----<br> Kenneth R. van Wyk<br> SC-L Moderator<br> KRvW Associates, LLC<br> <a href="http://www.KRvW.com">http://www.KRvW.com</a><br> <br> <br> <hr align="CENTER" size="3" width="95%"></span></font><font size="2"><font face="Consolas, Courier New, Courier"><span style="font-size:10pt">_______________________________________________<br> Secure Coding mailing list (SC-L) <a href="SC-L@securecoding.org">SC-L@securecoding.org</a><br> List information, subscriptions, etc - <a href="http://krvw.com/mailman/listinfo/sc-l">http://krvw.com/mailman/listinfo/sc-l</a><br> List charter available at - <a href="http://www.securecoding.org/list/charter.php">http://www.securecoding.org/list/charter.php</a><br> SC-L is hosted and moderated by KRvW Associates, LLC (<a href="http://www.KRvW.com">http://www.KRvW.com</a>)<br> as a free, non-commercial service to the software security community.<br> _______________________________________________<br> </span></font></font></blockquote> </div> </body></html>