|
Softpanorama |
May the source be with you, but remember the KISS principle ;-)
|
|
Software Engineering: A study akin to numerology and astrology, but lacking the precision of the former and the success of the latter. KISS Principle /kis' prin'si-pl/ n. "Keep It Simple, Stupid". A maxim often invoked when discussing design to fend off creeping featurism and control development complexity. Possibly related to the marketroid maxim on sales presentations, "Keep It Short and Simple". creeping featurism
/kree'ping fee'chr-izm/ n. [common]
1. Describes a systematic tendency to load more
chrome and
features onto systems at the expense of whatever elegance they may
have possessed when originally designed. See also
feeping creaturism. "You know, the main problem with
BSD Unix has always been creeping featurism." 2. More generally,
the tendency for anything complicated to become even more complicated
because people keep saying "Gee, it would be even better if it had this
feature too". (See
feature.) The result is usually a patchwork because it grew one
ad-hoc step at a time, rather than being planned. Planning is a lot
of work, but it's easy to add just one extra little feature to help
someone ... and then another ... and another... When creeping featurism
gets out of hand, it's like a cancer. Usually this term is used to describe
computer programs, but it could also be said of the federal government,
the IRS 1040 form, and new cars. A similar phenomenon sometimes afflicts
conscious redesigns; see
second-system effect. See also
creeping elegance. |
Software engineering (SE) has probably largest concentration of snake oil salesman after OO programming and software architecture is far from being an exclusion. Many published software methodologies/architectures claim to provide the benefits, that most of them can not deliver (UML is one good example). I see a lot of oversimplification of the real situation and unnecessary (and useless) formalisms. The main idea advocated here is simplification of software architecture (including usage of well-understood "Pipe and Filter model") and scripting languages.
There are a lot of quality general architectural resources available from the Net, therefore the list below represent only some links that I am interested personally. The stress here is on skepticism and this collection is neither complete, nor up to date. But still it might help students that are trying to study this complex and interesting subject. Or perhaps, if you already a software architect you might be able to expand your knowledge of the subject.
The primary purpose of Software Architecture courses is to teach students some higher level skills useful in designing and implementing complex software systems. In usually includes some information about classification (general and domain specific architectures), analysis and tools. As guys in Breadmear consulting aptly noted in their paper Who Software Architect Role:
A simplistic view of the role is that architects create architectures, and their responsibilities encompass all that is involved in doing so. This would include articulating the architectural vision, conceptualizing and experimenting with alternative architectural approaches, creating models and component and interface specification documents, and validating the architecture against requirements and assumptions.
However, any experienced architect knows that the role involves not just these technical activities, but others that are more political and strategic in nature on the one hand, and more like those of a consultant, on the other. A sound sense of business and technical strategy is required to envision the "right" architectural approach to the customer's problem set, given the business objectives of the architect's organization. Activities in this area include the creation of technology roadmaps, making assertions about technology directions and determining their consequences for the technical strategy and hence architectural approach.
Further, architectures are seldom embraced without considerable challenges from many fronts. The architect thus has to shed any distaste for what may be considered "organizational politics", and actively work to sell the architecture to its various stakeholders, communicating extensively and working networks of influence to ensure the ongoing success of the architecture.
But "buy-in" to the architecture vision is not enough either. Anyone involved in implementing the architecture needs to understand it. Since weighty architectural documents are notorious dust-gatherers, this involves creating and teaching tutorials and actively consulting on the application of the architecture, and being available to explain the rationale behind architectural choices and to make amendments to the architecture when justified.
Lastly, the architect must lead--the architecture team, the developer community, and, in its technical direction, the organization.
Again, I would like to stress that the main principle of software architecture is simple and well known -- it's famous KISS principle. While principle is simple its implementation is not and a lot of developers (especially developers with limited resources) paid dearly for violating this principle. I have found one one reference on simplicity in SE: R. S. Pressman. Simplicity. In Software Engineering, A Practitioner's Approach, page 452. McGraw Hill, 1997. Here open source tools can help because for those tools a complexity is not such a competitive advantage as for closed source tools.
I appreciate am architecture of software system that lead to small size implementations with simple, Spartan interface. In these days the usage of scripting languages can cut the volume of code more than in half in comparison with Java. That's why this site is advocating usage of scripting languages for complex software projects.
"Real Beauty can be found in Simplicity," and as you may know already, ' "Less" sometimes equal "More".' I continue to adhere to that philosophy. If you, too, have an eye for simplicity in software engineering, then you might benefit from this collection of links.
I think writing a good software system is somewhat similar to writing a multivolume series of books. Most writers will rewrite each chapter of book several times and changes general structure a lot. Rewriting large systems is more difficult, but also very beneficial. It make sense always consider the current version of the system a draft that can be substantially improved and simplified by discovering some new unifying and simplifying paradigm. Sometimes you can take a wrong direction, but still "nothing venture nothing have."
On a subsystem level a decent configuration management system can help going back. Too often people try to write and debug their fundamentally flawed architecturally "first draft", when it would have been much simpler and faster to rewrite it based on better understanding of architecture and better understanding of the problem. Actually rewriting can save time spend in debugging of the old version. That way, when you're done, you may get easy-to-understand, simple software systems, instead of just systems that "seems to work okay" (only as correct as your testing).
An interesting note about programming as art form can be found at The GNU-Linux Art Farm - 08-30-99
On component level refactoring (see Refactoring: Improving the Design of Existing Code) might be a useful simplification technique. Actually rewriting is a simpler term, but let's assume that refactoring is rewriting with some ideological frosting ;-). See Slashdot Book Reviews Refactoring Improving the Design of Existing Code.
I have found one reference on simplicity in SE: R. S. Pressman. Simplicity. In Software Engineering, A Practitioner's Approach, page 452. McGraw Hill, 1997.
Another relevant work (he try to promote his own solution -- you can skip this part) is the critique of "the technology mud slide" in a pretty interesting book by Harvard Business School Professor Clayton M. Christensen is very relevant. He defined "technology mudslide" very similar to Brooks "software development tar pit" -- a perpetual cycle of abandonment or retooling of existing systems in pursuit of the latest fashinable technology trend -- a cycle in which
"Coping with the relentless onslaught of technology change was akin to trying to climb a mudslide raging down a hill. You have to scramble with everything you've got to stay on top of it. and if you ever once stop to catch your breath, you get buried."
The complexity caused by adopting new technology for the sake of new technology are further exacerbated by the narrow focus and inexperience of many project leaders -- inexperience with mission-critical systems, systems of scale, software development disciplines, and project management. A Standish Group International survey recently showed that 46% of IT projects were over budget and overdue -- and 28% failed altogether. That's normal and probably the real failures figures are higher: great software managers and architects are rare and it is those people who determine the success of a software project.
Dr. Nikolai Bezroukov
|
|||||||
Andrew Binstock and Donald Knuth converse on the success of open source, the problem with multicore architecture, the disappointing lack of interest in literate programming, the menace of reusable code, and that urban legend about winning a programming contest with a single compilation.
Andrew Binstock: You are one of the fathers of the open-source revolution, even if you aren’t widely heralded as such. You previously have stated that you released TeX as open source because of the problem of proprietary implementations at the time, and to invite corrections to the code—both of which are key drivers for open-source projects today. Have you been surprised by the success of open source since that time?
Donald Knuth: The success of open source code is perhaps the only thing in the computer field that hasn’t surprised me during the past several decades. But it still hasn’t reached its full potential; I believe that open-source programs will begin to be completely dominant as the economy moves more and more from products towards services, and as more and more volunteers arise to improve the code.
For example, open-source code can produce thousands of binaries, tuned perfectly to the configurations of individual users, whereas commercial software usually will exist in only a few versions. A generic binary executable file must include things like inefficient "sync" instructions that are totally inappropriate for many installations; such wastage goes away when the source code is highly configurable. This should be a huge win for open source.
Yet I think that a few programs, such as Adobe Photoshop, will always be superior to competitors like the Gimp—for some reason, I really don’t know why! I’m quite willing to pay good money for really good software, if I believe that it has been produced by the best programmers.
Remember, though, that my opinion on economic questions is highly suspect, since I’m just an educator and scientist. I understand almost nothing about the marketplace.
Andrew: A story states that you once entered a programming contest at Stanford (I believe) and you submitted the winning entry, which worked correctly after a single compilation. Is this story true? In that vein, today’s developers frequently build programs writing small code increments followed by immediate compilation and the creation and running of unit tests. What are your thoughts on this approach to software development?
Donald: The story you heard is typical of legends that are based on only a small kernel of truth. Here’s what actually happened: John McCarthy decided in 1971 to have a Memorial Day Programming Race. All of the contestants except me worked at his AI Lab up in the hills above Stanford, using the WAITS time-sharing system; I was down on the main campus, where the only computer available to me was a mainframe for which I had to punch cards and submit them for processing in batch mode. I used Wirth’s ALGOL W system (the predecessor of Pascal). My program didn’t work the first time, but fortunately I could use Ed Satterthwaite’s excellent offline debugging system for ALGOL W, so I needed only two runs. Meanwhile, the folks using WAITS couldn’t get enough machine cycles because their machine was so overloaded. (I think that the second-place finisher, using that "modern" approach, came in about an hour after I had submitted the winning entry with old-fangled methods.) It wasn’t a fair contest.
As to your real question, the idea of immediate compilation and "unit tests" appeals to me only rarely, when I’m feeling my way in a totally unknown environment and need feedback about what works and what doesn’t. Otherwise, lots of time is wasted on activities that I simply never need to perform or even think about. Nothing needs to be "mocked up."
Andrew: One of the emerging problems for developers, especially client-side developers, is changing their thinking to write programs in terms of threads. This concern, driven by the advent of inexpensive multicore PCs, surely will require that many algorithms be recast for multithreading, or at least to be thread-safe. So far, much of the work you’ve published for Volume 4 of The Art of Computer Programming (TAOCP) doesn’t seem to touch on this dimension. Do you expect to enter into problems of concurrency and parallel programming in upcoming work, especially since it would seem to be a natural fit with the combinatorial topics you’re currently working on?
Donald: The field of combinatorial algorithms is so vast that I’ll be lucky to pack its sequential aspects into three or four physical volumes, and I don’t think the sequential methods are ever going to be unimportant. Conversely, the half-life of parallel techniques is very short, because hardware changes rapidly and each new machine needs a somewhat different approach. So I decided long ago to stick to what I know best. Other people understand parallel machines much better than I do; programmers should listen to them, not me, for guidance on how to deal with simultaneity.
Andrew: Vendors of multicore processors have expressed frustration at the difficulty of moving developers to this model. As a former professor, what thoughts do you have on this transition and how to make it happen? Is it a question of proper tools, such as better native support for concurrency in languages, or of execution frameworks? Or are there other solutions?
Donald: I don’t want to duck your question entirely. I might as well flame a bit about my personal unhappiness with the current trend toward multicore architecture. To me, it looks more or less like the hardware designers have run out of ideas, and that they’re trying to pass the blame for the future demise of Moore’s Law to the software writers by giving us machines that work faster only on a few key benchmarks! I won’t be surprised at all if the whole multithreading idea turns out to be a flop, worse than the "Titanium" approach that was supposed to be so terrific—until it turned out that the wished-for compilers were basically impossible to write.
Let me put it this way: During the past 50 years, I’ve written well over a thousand programs, many of which have substantial size. I can’t think of even five of those programs that would have been enhanced noticeably by parallelism or multithreading. Surely, for example, multiple processors are no help to TeX.[1]
How many programmers do you know who are enthusiastic about these promised machines of the future? I hear almost nothing but grief from software people, although the hardware folks in our department assure me that I’m wrong.
I know that important applications for parallelism exist—rendering graphics, breaking codes, scanning images, simulating physical and biological processes, etc. But all these applications require dedicated code and special-purpose techniques, which will need to be changed substantially every few years.
Even if I knew enough about such methods to write about them in TAOCP, my time would be largely wasted, because soon there would be little reason for anybody to read those parts. (Similarly, when I prepare the third edition of Volume 3 I plan to rip out much of the material about how to sort on magnetic tapes. That stuff was once one of the hottest topics in the whole software field, but now it largely wastes paper when the book is printed.)
The machine I use today has dual processors. I get to use them both only when I’m running two independent jobs at the same time; that’s nice, but it happens only a few minutes every week. If I had four processors, or eight, or more, I still wouldn’t be any better off, considering the kind of work I do—even though I’m using my computer almost every day during most of the day. So why should I be so happy about the future that hardware vendors promise? They think a magic bullet will come along to make multicores speed up my kind of work; I think it’s a pipe dream. (No—that’s the wrong metaphor! "Pipelines" actually work for me, but threads don’t. Maybe the word I want is "bubble.")
From the opposite point of view, I do grant that web browsing probably will get better with multicores. I’ve been talking about my technical work, however, not recreation. I also admit that I haven’t got many bright ideas about what I wish hardware designers would provide instead of multicores, now that they’ve begun to hit a wall with respect to sequential computation. (But my MMIX design contains several ideas that would substantially improve the current performance of the kinds of programs that concern me most—at the cost of incompatibility with legacy x86 programs.)
Andrew: One of the few projects of yours that hasn’t been embraced by a widespread community is literate programming. What are your thoughts about why literate programming didn’t catch on? And is there anything you’d have done differently in retrospect regarding literate programming?
Donald: Literate programming is a very personal thing. I think it’s terrific, but that might well be because I’m a very strange person. It has tens of thousands of fans, but not millions.
In my experience, software created with literate programming has turned out to be significantly better than software developed in more traditional ways. Yet ordinary software is usually okay—I’d give it a grade of C (or maybe C++), but not F; hence, the traditional methods stay with us. Since they’re understood by a vast community of programmers, most people have no big incentive to change, just as I’m not motivated to learn Esperanto even though it might be preferable to English and German and French and Russian (if everybody switched).
Jon Bentley probably hit the nail on the head when he once was asked why literate programming hasn’t taken the whole world by storm. He observed that a small percentage of the world’s population is good at programming, and a small percentage is good at writing; apparently I am asking everybody to be in both subsets.
Yet to me, literate programming is certainly the most important thing that came out of the TeX project. Not only has it enabled me to write and maintain programs faster and more reliably than ever before, and been one of my greatest sources of joy since the 1980s—it has actually been indispensable at times. Some of my major programs, such as the MMIX meta-simulator, could not have been written with any other methodology that I’ve ever heard of. The complexity was simply too daunting for my limited brain to handle; without literate programming, the whole enterprise would have flopped miserably.
If people do discover nice ways to use the newfangled multithreaded machines, I would expect the discovery to come from people who routinely use literate programming. Literate programming is what you need to rise above the ordinary level of achievement. But I don’t believe in forcing ideas on anybody. If literate programming isn’t your style, please forget it and do what you like. If nobody likes it but me, let it die.
On a positive note, I’ve been pleased to discover that the conventions of CWEB are already standard equipment within preinstalled software such as Makefiles, when I get off-the-shelf Linux these days.
Andrew: In Fascicle 1 of Volume 1, you reintroduced the MMIX computer, which is the 64-bit upgrade to the venerable MIX machine comp-sci students have come to know over many years. You previously described MMIX in great detail in MMIXware. I’ve read portions of both books, but can’t tell whether the Fascicle updates or changes anything that appeared in MMIXware, or whether it’s a pure synopsis. Could you clarify?
Donald: Volume 1 Fascicle 1 is a programmer’s introduction, which includes instructive exercises and such things. The MMIXware book is a detailed reference manual, somewhat terse and dry, plus a bunch of literate programs that describe prototype software for people to build upon. Both books define the same computer (once the errata to MMIXware are incorporated from my website). For most readers of TAOCP, the first fascicle contains everything about MMIX that they’ll ever need or want to know.
I should point out, however, that MMIX isn’t a single machine; it’s an architecture with almost unlimited varieties of implementations, depending on different choices of functional units, different pipeline configurations, different approaches to multiple-instruction-issue, different ways to do branch prediction, different cache sizes, different strategies for cache replacement, different bus speeds, etc. Some instructions and/or registers can be emulated with software on "cheaper" versions of the hardware. And so on. It’s a test bed, all simulatable with my meta-simulator, even though advanced versions would be impossible to build effectively until another five years go by (and then we could ask for even further advances just by advancing the meta-simulator specs another notch).
Suppose you want to know if five separate multiplier units and/or three-way instruction issuing would speed up a given MMIX program. Or maybe the instruction and/or data cache could be made larger or smaller or more associative. Just fire up the meta-simulator and see what happens.
Andrew: As I suspect you don’t use unit testing with MMIXAL, could you step me through how you go about making sure that your code works correctly under a wide variety of conditions and inputs? If you have a specific work routine around verification, could you describe it?
Donald: Most examples of machine language code in TAOCP appear in Volumes 1-3; by the time we get to Volume 4, such low-level detail is largely unnecessary and we can work safely at a higher level of abstraction. Thus, I’ve needed to write only a dozen or so MMIX programs while preparing the opening parts of Volume 4, and they’re all pretty much toy programs—nothing substantial. For little things like that, I just use informal verification methods, based on the theory that I’ve written up for the book, together with the MMIXAL assembler and MMIX simulator that are readily available on the Net (and described in full detail in the MMIXware book).
That simulator includes debugging features like the ones I found so useful in Ed Satterthwaite’s system for ALGOL W, mentioned earlier. I always feel quite confident after checking a program with those tools.
Andrew: Despite its formulation many years ago, TeX is still thriving, primarily as the foundation for LaTeX. While TeX has been effectively frozen at your request, are there features that you would want to change or add to it, if you had the time and bandwidth? If so, what are the major items you add/change?
Donald: I believe changes to TeX would cause much more harm than good. Other people who want other features are creating their own systems, and I’ve always encouraged further development—except that nobody should give their program the same name as mine. I want to take permanent responsibility for TeX and Metafont, and for all the nitty-gritty things that affect existing documents that rely on my work, such as the precise dimensions of characters in the Computer Modern fonts.
Andrew: One of the little-discussed aspects of software development is how to do design work on software in a completely new domain. You were faced with this issue when you undertook TeX: No prior art was available to you as source code, and it was a domain in which you weren’t an expert. How did you approach the design, and how long did it take before you were comfortable entering into the coding portion?
Donald: That’s another good question! I’ve discussed the answer in great detail in Chapter 10 of my book Literate Programming, together with Chapters 1 and 2 of my book Digital Typography. I think that anybody who is really interested in this topic will enjoy reading those chapters. (See also Digital Typography Chapters 24 and 25 for the complete first and second drafts of my initial design of TeX in 1977.)
Andrew: The books on TeX and the program itself show a clear concern for limiting memory usage—an important problem for systems of that era. Today, the concern for memory usage in programs has more to do with cache sizes. As someone who has designed a processor in software, the issues of cache-aware and cache-oblivious algorithms surely must have crossed your radar screen. Is the role of processor caches on algorithm design something that you expect to cover, even if indirectly, in your upcoming work?
Donald: I mentioned earlier that MMIX provides a test bed for many varieties of cache. And it’s a software-implemented machine, so we can perform experiments that will be repeatable even a hundred years from now. Certainly the next editions of Volumes 1-3 will discuss the behavior of various basic algorithms with respect to different cache parameters.
In Volume 4 so far, I count about a dozen references to cache memory and cache-friendly approaches (not to mention a "memo cache," which is a different but related idea in software).
Andrew: What set of tools do you use today for writing TAOCP? Do you use TeX? LaTeX? CWEB? Word processor? And what do you use for the coding?
Donald: My general working style is to write everything first with pencil and paper, sitting beside a big wastebasket. Then I use Emacs to enter the text into my machine, using the conventions of TeX. I use tex, dvips, and gv to see the results, which appear on my screen almost instantaneously these days. I check my math with Mathematica.
I program every algorithm that’s discussed (so that I can thoroughly understand it) using CWEB, which works splendidly with the GDB debugger. I make the illustrations with MetaPost (or, in rare cases, on a Mac with Adobe Photoshop or Illustrator). I have some homemade tools, like my own spell-checker for TeX and CWEB within Emacs. I designed my own bitmap font for use with Emacs, because I hate the way the ASCII apostrophe and the left open quote have morphed into independent symbols that no longer match each other visually. I have special Emacs modes to help me classify all the tens of thousands of papers and notes in my files, and special Emacs keyboard shortcuts that make bookwriting a little bit like playing an organ. I prefer rxvt to xterm for terminal input. Since last December, I’ve been using a file backup system called backupfs, which meets my need beautifully to archive the daily state of every file.
According to the current directories on my machine, I’ve written 68 different CWEB programs so far this year. There were about 100 in 2007, 90 in 2006, 100 in 2005, 90 in 2004, etc. Furthermore, CWEB has an extremely convenient "change file" mechanism, with which I can rapidly create multiple versions and variations on a theme; so far in 2008 I’ve made 73 variations on those 68 themes. (Some of the variations are quite short, only a few bytes; others are 5KB or more. Some of the CWEB programs are quite substantial, like the 55-page BDD package that I completed in January.) Thus, you can see how important literate programming is in my life.
I currently use Ubuntu Linux, on a standalone laptop—it has no Internet connection. I occasionally carry flash memory drives between this machine and the Macs that I use for network surfing and graphics; but I trust my family jewels only to Linux. Incidentally, with Linux I much prefer the keyboard focus that I can get with classic FVWM to the GNOME and KDE environments that other people seem to like better. To each his own.
Andrew: You state in the preface of Fascicle 0 of Volume 4 of TAOCP that Volume 4 surely will comprise three volumes and possibly more. It’s clear from the text that you’re really enjoying writing on this topic. Given that, what is your confidence in the note posted on the TAOCP website that Volume 5 will see light of day by 2015?
Donald: If you check the Wayback Machine for previous incarnations of that web page, you will see that the number 2015 has not been constant.
You’re certainly correct that I’m having a ball writing up this material, because I keep running into fascinating facts that simply can’t be left out—even though more than half of my notes don’t make the final cut.
Precise time estimates are impossible, because I can’t tell until getting deep into each section how much of the stuff in my files is going to be really fundamental and how much of it is going to be irrelevant to my book or too advanced. A lot of the recent literature is academic one-upmanship of limited interest to me; authors these days often introduce arcane methods that outperform the simpler techniques only when the problem size exceeds the number of protons in the universe. Such algorithms could never be important in a real computer application. I read hundreds of such papers to see if they might contain nuggets for programmers, but most of them wind up getting short shrift.
From a scheduling standpoint, all I know at present is that I must someday digest a huge amount of material that I’ve been collecting and filing for 45 years. I gain important time by working in batch mode: I don’t read a paper in depth until I can deal with dozens of others on the same topic during the same week. When I finally am ready to read what has been collected about a topic, I might find out that I can zoom ahead because most of it is eminently forgettable for my purposes. On the other hand, I might discover that it’s fundamental and deserves weeks of study; then I’d have to edit my website and push that number 2015 closer to infinity.
Andrew: In late 2006, you were diagnosed with prostate cancer. How is your health today?
Donald: Naturally, the cancer will be a serious concern. I have superb doctors. At the moment I feel as healthy as ever, modulo being 70 years old. Words flow freely as I write TAOCP and as I write the literate programs that precede drafts of TAOCP. I wake up in the morning with ideas that please me, and some of those ideas actually please me also later in the day when I’ve entered them into my computer.
On the other hand, I willingly put myself in God’s hands with respect to how much more I’ll be able to do before cancer or heart disease or senility or whatever strikes. If I should unexpectedly die tomorrow, I’ll have no reason to complain, because my life has been incredibly blessed. Conversely, as long as I’m able to write about computer science, I intend to do my best to organize and expound upon the tens of thousands of technical papers that I’ve collected and made notes on since 1962.
Andrew: On your website, you mention that the Peoples Archive recently made a series of videos in which you reflect on your past life. In segment 93, "Advice to Young People," you advise that people shouldn’t do something simply because it’s trendy. As we know all too well, software development is as subject to fads as any other discipline. Can you give some examples that are currently in vogue, which developers shouldn’t adopt simply because they’re currently popular or because that’s the way they’re currently done? Would you care to identify important examples of this outside of software development?
Donald: Hmm. That question is almost contradictory, because I’m basically advising young people to listen to themselves rather than to others, and I’m one of the others. Almost every biography of every person whom you would like to emulate will say that he or she did many things against the "conventional wisdom" of the day.
Still, I hate to duck your questions even though I also hate to offend other people’s sensibilities—given that software methodology has always been akin to religion. With the caveat that there’s no reason anybody should care about the opinions of a computer scientist/mathematician like me regarding software development, let me just say that almost everything I’ve ever heard associated with the term "extreme programming" sounds like exactly the wrong way to go...with one exception. The exception is the idea of working in teams and reading each other’s code. That idea is crucial, and it might even mask out all the terrible aspects of extreme programming that alarm me.
I also must confess to a strong bias against the fashion for reusable code. To me, "re-editable code" is much, much better than an untouchable black box or toolkit. I could go on and on about this. If you’re totally convinced that reusable code is wonderful, I probably won’t be able to sway you anyway, but you’ll never convince me that reusable code isn’t mostly a menace.
Here’s a question that you may well have meant to ask: Why is the new book called Volume 4 Fascicle 0, instead of Volume 4 Fascicle 1? The answer is that computer programmers will understand that I wasn’t ready to begin writing Volume 4 of TAOCP at its true beginning point, because we know that the initialization of a program can’t be written until the program itself takes shape. So I started in 2005 with Volume 4 Fascicle 2, after which came Fascicles 3 and 4. (Think of Star Wars, which began with Episode 4.)
A just-published IBM patent application for a Software Inspection Management Tool claims to improve software quality by taking a chess-clock-like approach to code walkthroughs. An inspection rate monitor with 'a pause button, a resume button, a complete button, a total lines inspected indication, and a total lines remaining to be inspected indication' keeps tabs on participants' progress and changes color when management's expectations — measured in lines per hour — are not being met."
Right Wally. so, you write line after line of utterly meaningless horseshit to keep your line count up, and then route the data to a place where it gets commented out... and then follow that with line after line of craptastic documentation. Copy paste the Unabomber's manifesto - whatever - as long as the line count counter keeps ticking a long.
You can probably get employee of the month for all the code you've written, and only in a few hours a day! Meanwhile, you're spending the rest of your time looking for work at a company that doesn't pull this kind of bullshit on its workers.
Re:what a stupid stupid idea (Score:5, Funny)
by arth1 (260657) on Saturday April 26, @10:50PM (#23211076) Homepage JournalSomeone I know was told to increase lines-of-code output by 20%, not counting comments. He immediately complied and exceeded expectations by switching to K&R style, declaring function variables on separate lines instead of inline, and placing { on a separate line.
He was ready to place semicolons on a separate line too, if management wanted even higher efficiency.
Metropolis.... (Score:3, Interesting)
by drolli (522659) on Saturday April 26, @10:41PM (#23211016)This reminds me of:
Metropolis (Fritz Lang, 1927). Excellent movie BTW.
The worker at the power plant collapses during a a shift which was too long and required to many new operations demanded by a clock-like device. The power plant nearly explodes, because he can not keed up with the pace of this clock-like device.
Rushing through the code tends to make us find fewer bugs per time spent-- if anything, it's better to more closely inspect a third of the code at a third of the pace and not inspect the other two-thirds at all than to inspect all of it in the same amount of time.
The issue being, we tend to want to get the inspection of the code over with; such a clock might act as a nice reminder to slow down a bit.
From what I can see, the implication that this has anything to do with management harassing the developers and testers is completely conjecture on the part of the slashdot submitter. The only context in which the word "manage" appears in the entire application is as part of the phrase "management tool", which to me implies that it's supposed to be entirely to help the testing and development staff. (Okay, there's one occurance which is "inspection process manager".)
I know that IBM has a famous history of having associated productivity with lines of code, but I really don't think they're being quite so dim-witted with this one. I haven't read the application in detail, but to me it looks more like someone's been developing a tool to help with code inspection. By the looks of it, it has a certain way of displaying the code, it has a method of recording noted defects and comments, and it has a feature of timing how long things are taking and how long a user is spending on certain parts of a code-base.
I can't see any direct implication in the patent application that this is primarily for management to measure staff performance to compare with pre-defined expectations. On the other hand I can see a lot of references in the patent application to the code inspector themselves using this tool to assist their work. I think it's much more likely that someone running an inspection could use such a tool to help them keep track of the most fragile parts of the code, and which areas are tying up the most of their time. If there was a deadline for inspection, it'd probably also help to highlight if you were spending far too much time in one place without having even reached other areas that might be important.
Whether it would work or be any use at all it another issue, but if it's a completely wacky idea then it wouldn't be the first that someone tried to patent. Many good patented ideas seemed silly or ridiculous before a working implementation was produced to demonstrate otherwise, but if an inventor had waited until it was clearly useful before patenting it, it'd be a lot harder.
Monday, June 26, 2006 | Interoperability Happens The Vietnam of Computer Science(Two years ago, at Microsoft's TechEd in San Diego, I was involved in a conversation at an after-conference event with Harry Pierson and Clemens Vasters, and as is typical when the three of us get together, architectural topics were at the forefront of our discussions. An crowd gathered around us, and it turned into an impromptu birds-of-a-feather session. The subject of object/relational mapping technologies came up, and it was there and then that I first coined the phrase, "Object/relational mapping is the Vietnam of Computer Science". In the intervening time, I've received numerous requests to flesh out the discussion behind that statement, and given Microsoft's recent announcement regarding "entity support" in ADO.NET 3.0 and the acceptance of the Java Persistence API as a replacement for both EJB Entity Beans and JDO, it seemed time to do exactly that.)
... ... ...
Given, then, that objects-to-relational mapping is a necessity in a modern enterprise system, how can anyone proclaim it a quagmire from which there is no escape? Again, Vietnam serves as a useful analogy here--while the situation in South Indochina required a response from the Americans, there were a variety of responses available to the Kennedy and Johson Administrations, including the same kind of response that the recent fall of Suharto in Malaysia generated from the US, which is to say, none at all. (Remember, Eisenhower and Dulles didn't consider South Indochina to be a part of the Domino Theory in the first place; they were far more concerned about Japan and Europe.)
Several possible solutions present themselves to the O/R-M problem, some requiring some kind of "global" action by the community as a whole, some more approachable to development teams "in the trenches":
Note that this list is not presented in any particular order; while some are more attractive to others, which are "better" is a value judgment that every developer and development team must make for themselves.
- Abandonment. Developers simply give up on objects entirely, and return to a programming model that doesn't create the object/relational impedance mismatch. While distasteful, in certain scenarios an object-oriented approach creates more overhead than it saves, and the ROI simply isn't there to justify the cost of creating a rich domain model. ([Fowler] talks about this to some depth.) This eliminates the problem quite neatly, because if there are no objects, there is no impedance mismatch.
- Wholehearted acceptance. Developers simply give up on relational storage entirely, and use a storage model that fits the way their languages of choice look at the world. Object-storage systems, such as the db4o project, solve the problem neatly by storing objects directly to disk, eliminating many (but not all) of the aforementioned issues; there is no "second schema", for example, because the only schema used is that of the object definitions themselves. While many DBAs will faint dead away at the thought, in an increasingly service-oriented world, which eschews the idea of direct data access but instead requires all access go through the service gateway thus encapsulating the storage mechanism away from prying eyes, it becomes entirely feasible to imagine developers storing data in a form that's much easier for them to use, rather than DBAs.
- Manual mapping. Developers simply accept that it's not such a hard problem to solve manually after all, and write straight relational-access code to return relations to the language, access the tuples, and populate objects as necessary. In many cases, this code might even be automatically generated by a tool examining database metadata, eliminating some of the principal criticism of this approach (that being, "It's too much code to write and maintain").
- Acceptance of O/R-M limitations. Developers simply accept that there is no way to efficiently and easily close the loop on the O/R mismatch, and use an O/R-M to solve 80% (or 50% or 95%, or whatever percentage seems appropriate) of the problem and make use of SQL and relational-based access (such as "raw" JDBC or ADO.NET) to carry them past those areas where an O/R-M would create problems. Doing so carries its own fair share of risks, however, as developers using an O/R-M must be aware of any caching the O/R-M solution does within it, because the "raw" relational access will clearly not be able to take advantage of that caching layer.
- Integration of relational concepts into the languages. Developers simply accept that this is a problem that should be solved by the language, not by a library or framework. For the last decade or more, the emphasis on solutions to the O/R problem have focused on trying to bring objects closer to the database, so that developers can focus exclusively on programming in a single paradigm (that paradigm being, of course, objects). Over the last several years, however, interest in "scripting" languages with far stronger set and list support, like Ruby, has sparked the idea that perhaps another solution is appropriate: bring relational concepts (which, at heart, are set-based) into mainstream programming languages, making it easier to bridge the gap between "sets" and "objects". Work in this space has thus far been limited, constrained mostly to research projects and/or "fringe" languages, but several interesting efforts are gaining visibility within the community, such as functional/object hybrid languages like Scala or F#, as well as direct integration into traditional O-O languages, such as the LINQ project from Microsoft for C# and Visual Basic. One such effort that failed, unfortunately, was the SQL/J strategy; even there, the approach was limited, not seeking to incorporate sets into Java, but simply allow for embedded SQL calls to be preprocessed and translated into JDBC code by a translator.
- Integration of relational concepts into frameworks. Developers simply accept that this problem is solvable, but only with a change of perspective. Instead of relying on language or library designers to solve this problem, developers take a different view of "objects" that is more relational in nature, building domain frameworks that are more directly built around relational constructs. For example, instead of creating a Person class that holds its instance data directly in fields inside the object, developers create a Person class that holds its instance data in a RowSet (Java) or DataSet (C#) instance, which can be assembled with other RowSets/DataSets into an easy-to-ship block of data for update against the database, or unpacked from the database into the individual objects.
Just as it's conceivable that the US could have achieved some measure of "success" in Vietnam had it kept to a clear strategy and understood a more clear relationship between commitment and results (ROI, if you will), it's conceivable that the object/relational problem can be "won" through careful and judicious application of a strategy that is celarly aware of its own limitations. Developers must be willing to take the "wins" where they can get them, and not fall into the trap of the Slippery Slope by looking to create solutions that increasingly cost more and yield less. Unfortunately, as the history of the Vietnam War shows, even an awareness of the dangers of the Slippery Slope is often not enough to avoid getting bogged down in a quagmire. Worse, it is a quagmire that is simply too attractive to pass up, a Siren song that continues to draw development teams from all sizes of corporations (including those at Microsoft, IBM, Oracle, and Sun, to name a few) against the rocks, with spectacular results. Lash yourself to the mast if you wish to hear the song, but let the sailors row.
Endnotes
1 Later analysis by the principals involved--including then-Secretary of Defense Robert McNamara--concluded that half of the attack never even took place.
2 It is perhaps the greatest irony of the war, that the man Fate selected to lead during America's largest foreign entanglement was a leader whose principal focus was entirely aimed within his own shores. Had circumstances not conspired otherwise, the hippies chanting "Hey, hey LBJ, how many boys did you kill today" outside the Oval Office could very well have been Johnson's staunchest supporters.
3 Ironically, encapsulation, for purposes of maintenance simplicity, turns out to be a major motivation for almost all of the major innovations in Linguistic Computer Science--procedural, functional, object, aspect, even relational technologies ([Date02]) and other languages all cite "encapsulation" as major driving factors.
4 We could, perhaps, consider stored procedure languages like T-SQL or PL/SQL to be "relational" programming languages, but even then, it's extremely difficult to build a UI in PL/SQL.
5 In this case, I was measuring Java RMI method calls against local method calls. Similar results are pretty easily obtainable for SQL-based data access by measuring out-of-process calls against in-process calls using a database product that supports both, such as Cloudscape/Derby or HSQL (Hypersonic SQL).
References
[Fussell]: Foundations of Object Relational Mapping, by Mark L. Fussell, v0.2 (mlf-970703)
[Fowler] Patterns of Enterprise Application Architecture, by Martin Fowler
[Date04]: Introduction to Database Systems, 8th Edition, by Chris Date.
[Neward04]: Effective Enterprise Java
Nov 9, 2007 | InformITIn the late 1990s, during the first dotcom bubble, there was a perception that a computer science degree was a quick way of making money. The dotcom boom had venture capitalists throwing money at the craziest schemes, just because they happened to involve the Internet. While not entirely grounded in fact, this trend led to a perception that anyone walking out of a university with a computer science degree would immediately find his pockets full of venture capital funding.
Then came the inevitable crash, and suddenly there were a lot more IT professionals than IT jobs. Many of these people were the ones that just got into the industry to make a quick buck, but quite a few were competent people now unemployed. This situation didn’t do much for the perception of computer science as an attractive degree scheme.
Since the end of the first dotcom bubble, we’ve seen a gradual decline in the number of people applying to earn computer science degrees. In the UK, many departments were able to prop up the decline in local applicants by attracting more overseas students, particularly from Southeast Asia, by dint of being considerably cheaper than American universities for those students wishing to study abroad. This only slowed the drop, however, and some people are starting to ask whether computer science is dying.
Computer Science and Telescopes
Part of the problem is a lack of understanding of exactly what computer science is. Even undergraduates accepted into computer science courses generally have only the broadest idea of what the subject entails. It’s hardly surprising, then, that people would wonder if the discipline is dying.
Even among those in computing-related fields, there’s a general feeling that computer science is basically a vocational course, teaching programming. In January 2007, the British Computer Society (BCS) published an article by Neil McBride of De Montfort University, entitled "The Death of Computing." Although the content was of a lower quality than the average Slashdot troll post (which at least tries to pretend that it’s raising a valid point) and convinced me that I didn’t want to be a member of the BCS, it was nevertheless circulated quite widely. This article contained choice lines such as the following: "What has changed is the need to know low-level programming or any programming at all. Who needs C when there’s Ruby on Rails?"
Who needs C? Well, at least those people who want to understand something of what’s going on when the Ruby on Rails program runs. An assembly language or two would do equally well. The point of an academic degree, as opposed to a vocational qualification, is to teach understanding, rather than skills—a point sadly lost on Dr. McBride when he penned his article.
In attempting to describe computer science, Edsger Dijkstra claimed, "Computer science is no more about computers than astronomy is about telescopes." I like this quote, but it’s often taken in the wrong way by people who haven’t met many astronomers. When I was younger, I was quite interested in astronomy, and spent a fair bit of time hanging around observatories and reading about the science (as well as looking through telescopes). During this period, I learned a lot more about optics than I ever did in physics courses at school. I never built my own telescope, but a lot of real astronomers did, and many of the earliest members of the profession made considerable contributions to our understanding of optics.
There’s a difference between a telescope builder and an astronomer, of course. A telescope builder is likely to know more about the construction of telescopes and less about the motion of stellar bodies. But both will have a solid understanding of what happens to light as it travels through the lenses and bounces off the mirrors. Without this understanding, astronomy is very difficult.
The same principle holds true for computer science. A computer scientist may not fabricate her own ICs, and may not write her own compiler and operating system. In the modern age, these things are generally too complicated for a single person to do to a standard where the result can compete with off-the-shelf components. But the computer scientist definitely will understand what’s happening in the compiler, operating system, and CPU when a program is compiled and run.
A telescope is an important tool to an astronomer, and a computer is an important tool for a computer scientist—but each is merely a tool, not the focus of study. For an astronomer, celestial bodies are studied using a telescope. For a computer scientist, algorithms are studied using a computer.
Software and hardware are often regarded as being very separate concepts. This is a convenient distinction, but it’s not based on any form of reality. The first computers had no software per se, and needed to be rewired to run different programs. Modern hardware often ships with firmware—software that’s closely tied to the hardware to perform special-purpose functions on general-purpose silicon. Whether a task is handled in hardware or software is of little importance from a scientific perspective. (From an engineering perspective, there are tradeoffs among cost, maintenance, and speed.) Either way, the combination of hardware and software is a concrete instantiation of an algorithm, allowing it to be studied.
As with other subjects, there are a lot of specializations within computer science. I tend to view the subject as the intersection between three fields:
- Mathematics
- Engineering
- Psychology
At the very mathematical end are computer scientists who study algorithms without the aid of a computer, purely in the abstract. Closer to engineering are those who build large hardware and software systems. In between are the people who use formal verification tools to construct these systems.
A computer isn’t much use without a human instructing it, and this is where the psychology is important. Computers need to interact with humans a lot, and neither group is really suited to the task. The reason that computers have found such widespread use is that they perform well in areas where humans perform poorly (and vice versa). Trying to find a mechanism for describing something that is understandable by both humans and computers is the role of the "human/computer interaction" (HCI) subdiscipline within computer science. This is generally close to psychology.
HCI isn’t the only part of computer science related to psychology. As far back as 1950, Alan Turing proposed the Turing Test as a method of determining whether an entity should be treated as intelligent.
It’s understandable that people who aren’t directly exposed to computer science would miss the breadth of the discipline, associating it with something more familiar. One solution proposed for this lack of vision is that of renaming the subject to "informatics." In principle, this is a good idea, but the drawback is that it’s very difficult to describe someone as an "informatician" with a straight face.
A software developer must be part writer and poet, part salesperson and public speaker, part artist and designer, and always equal parts logic and empathy. The process of developing software differs from organization to organization. Some are more "shoot from the hip" style, others, like my current employer are much more careful and deliberate. In my 8 years of experience I've worked for 4 different companies, each with their own process. But out of all of them, I've found these stages to be universally applicable:
Dreaming and Shaping
A piece of software starts, before any code is written, as an idea or as a problem to be solved. Its a constraint on a plant floor, a need for information, a better way to work, a way to communicate, or a way to play. It is always tied to a human being ≈ their job, their entertainment┘ their needs. A good process will explore this driving factor well. In the project I'm wrapping up now I felt strongly, and my employer agreed with me, that to understand what we needed to do, we'd have to go to the customer and feel their pain. We'd have to watch them work so we could understand their constraints. And we'd have to explore the other solutions out there to the problem we were trying to solve.
Once you understand what you need to build, you still don't begin building it. Like an architect or a designer, you start with a sketch, and you create a design. In software your design is expressed in documents and in diagrams. Its not uncommon for the design process to take longer than the coding process. As a part of your design, you have to understand your tools. Imagine an author who, at the start of each book, needs to research every writing instrument on the market first. You have to become knowledgeable about the strengths and weaknesses of each tool out there, because your choice of instrument, as much as your design or skill as a programmer, can impact the success of your work. Then you review. With marketing and with every subject matter expert and team member you can find who will have any advice to give. You meet and you discuss and you refine your design, your pre-conceptions, and even your selected tools until it passes the most intense scrutiny.
Once you have these things down, you have to be willing to give them up. You have to go back to the customer, or the originator of the problem, and sell them your solution. You put on a sales hat and you pitch what you've dreamt up┘ then wait with bated breath while they dissect your brain child. If you've understood them, and the problem, you'll only need to make adjustments or adapt to information you didn't previously have. Always you need to anticipate changes you didn't plan for ≈ they'll come at you through-out the project.
Once you know how the solution is going to work, or sometimes even before then, you need to figure out how people are going to work with your solution. Software that can't be understood can't be used, so no matter how brilliant your design, if your interface isn't elegant and beautiful and intuitive, your project is a failure.
I don't pick those adjectives lightly either. All of them are required, in balance. If its not elegant, then its wasteful and you'll likely need to find a new job. If its not beautiful, then no one will want to use it. And if its not intuitive, no one will be able to use it. The attention to detail required of a good interface developer is on par with that of a good painter. Every dot, every stroke, every color choice is significant.
To make something easy to use requires at least a basic understanding of human reactions, an awareness of cognitive norms. People react to your software, often on a very base level. If you don't believe me, think of the last time your computer crashed before you had a chance to save the last 2 hours worth of an essay, or a game you were playing. What you put before your users must be easy to look at so that they are comfortable learning it. It must anticipate their needs so that they don't get frustrated. It must suggest its use, simply by being on the screen. And above all else, it must preserve their focus and their effort.
So you paint, using PowerPoint, or Visio, or some other tool, your picture of what you think the customer is going to want to use, and once again you don your sales hat and try to sell it to them. Only, unlike a salesperson selling someone else's product, you are selling your own work, and are inevitably emotionally-attached to it. Still, you know criticism is good, because it makes the results better, so you force yourself to be logical about it.
Then finally, when your solution is approved, and your interface is understood, you can move on to the really fun part of your job:
Prose and Poetry
A good sonnet isn't only identified by the letters or words on the page, but by the cadence, the meter, the measure, the flow┘ a good piece of literature is beautiful because it is shaped carefully yet communicates eloquently.
Code is no different. The purpose of code is to express a solution. A project consists of small stanzas, called "Methods" or "Functions" depending on what language you use. Each of these verses must be constructed in such a way that it is efficient, tightly-crafted, and effective. And like a poem, there are rules that dictate how it should be shaped. There is beauty in a clever Function.
But the real beauty of code goes further than poetry. Because it re-uses itself. Maybe its more like music, where a particular measure is repeated later in the song, and through its familiarity, it adds to the shape of the whole piece. Functions are like that, in that they're called throughout the software. Sometimes they repeat within themselves, in iterations, like the repeating patterns you see in nature. fern.jpgAnd when the pieces are added up, each in itself a little work of art, they make, if programmed properly, a whole that is much more than a sum. Its is an intertwined, and constantly moving piece of art.
As programmers, we add things called "log messages" so that we can see these parts working together, because without this output, the flow of the data through the different rungs and branches we put together is so fluid that we can't even observe it and, like trying to fathom the number of stars in the sky, it is difficult to even conceptualize visually the thousands of interactions a second that your code is causing. And we need to do this, because next comes a Quality Assurance Engineer (or QA) who tries to break your code, question your decisions, and generally force you to do better than what you thought was your best.
I truly believe that code is an art form. One that only a small portion of the population can appreciate. Sure anyone can walk into the Louvre and appreciate the end result of a Davinci or a Van Gogh, but only a true artist or student of art can really understand the intricacy of the work behind it. Similarly, most people can recognize a good piece of software when they use it (certainly anyone can recognize a bad piece of software) but it takes a true artist, or at least an earnest student, to understand just how brilliant ≈ or how wretched ≈ the work behind it is.
And always, as you weave your code, you have to be prepared to change it, to re-use it, to re-contain it, to re-purpose it in ways that you can't have planned for. Because that is the nature of your art form ≈ always changing and advancing.
Publishing and Documenting
Its been said that a scientist or researcher must "publish or perish." The same is true of a software developer. A brilliant piece of code, if not used, is lost. Within months it will become obsolete, or replaced, or usurped, and your efforts will become meaningless, save for the satisfaction of having solved a problem on your own.
So after months of wearing jeans, chugging caffeine, cluttering your desk with sketches and reference material, you clean yourself up, put on a nice pair of pants, comb your hair, and sell again. Although most organizations have a sales force and a marketing department, a savvy customer will invariably want technical details that a non-coder can't supply. As a lead developer on a project, it falls to you to instill confidence, to speak articulately and passionately about the appropriateness and worth of your solution. Again, as before, pride is a weakness here, because no matter how good you are, someone will always ask if your software can do something it can't ≈ user's are never really satisfied. So you think back to the design process, you remind them when they had a part in the decisions, and you attempt to impress upon them respect for the solution you have now, while acknowledging that there will always be a version 2.0.
And you write and you teach. Not so much in my current job, but in one previous, as a lead developer it was my responsibility to educate people on the uses of our technology ≈ to come up with ways to express the usefulness of a project without boring people with too many technical details. One of the best parts of software development, a part that I miss since its not within my present job description, is getting up in front of people ≈ once they've accepted your solution ≈ and teaching them how to use it and apply it. Taking them beyond the basic functionality and showing them the tricks and shortcuts and advanced features that you programmed in, not because anyone asked you for them, but because you knew in your gut they should be there.
And Repeat┘
Then there's a party, a brief respite, where you celebrate your victory, congratulate those who've worked on parallel projects, and express your deepest gratitude for your peers who've lent their own particular area of expertise to your project┘ And you start again. Because like I'm sure any sports team feels, you are only as good as your latest victory.
So do I fix computers? Often its easier or more expedient to hack together a solution to a problem on my own ≈ certainly the I.T. Department is becoming something of a slow-moving dinosaur in an age where computers aren't the size of buildings, and most of us are comfortable re-installing Windows on our own ≈ but that's not a part of my job description.
No, I, like my peers, produce art. Functional, useful, but still beautiful, art. We are code poets, and it is our prose that builds the tools people use every day.
However, unlike most other artists, we're usually paid pretty well for our work ;-)Slashdot The Life of a Software Engineer
Obligatory Futurama Quote (Score:4, Interesting)
by ari_j (90255) on Monday February 04, @03:28PM (#22296856) HomepageJonathan Wise writes to share with us an interesting bit of prose describing life as a software engineer.Interesting. No, wait, that other thing. Tedious.
A software developer must be part writer and poet, part salesperson and public speaker, part artist and designer, and always equal parts logic and empathy.No. You're a code monkey. Your poetry sucks, your writing sucks (to the point of your having written "its for the best"), your salesmanship sucks (your article sells nobody on anything), your public speaking is probably twice as melodramatic as your article and therefore sucks, your art sucks, your designs are plagued with whining about having to make them, and you have never experienced logic or empathy as long as you've lived.
Get over it. If you want to make your pathetic job seem more important than it is, Slashdot isn't the place to do it. If you want to whine about being a code monkey, Slashdot is even less appropriate of a place to do it. Just get up, get coffee, go to job, and have boring meeting with boring manager Rob.
Poetry and Prose... (Score:2)
by flajann (658201) on Monday February 04, @04:55PM (#22298404) Homepage JournalI can tell many of the posters did not bother to read the full article. Perhaps your excuse is because his site crumbled under the "slash-dot effect", but still.Having actually read what Jonathan Wise wrote, I thought he made quite a few good salient points. Going from idea to finished product is as much about art as it is about science. There is artistry involved at many different levels. Alas, the end-user only gets to directly see the top layers of that art. The actual organization of the code, the algorithms used, to optimizations, the kludges -- if any! --, the language constructs exploited, the database schema, if applicable, all add to the art and elegance of a software.
Most of the beauty will forever lie hidden from any but those who dive into and interact directly with the code itself. But the end-user will be presented with the form and function, and perhaps can have a appreciation for the art behind the art.
Perhaps another term for what we do, which embraces all aspects of creating software, is hacker. To the cognoscenti who appreciates the true meaning of that term and not the disparaging, derogatory version the silly media created, "hacker" says it all. And is a greater thing than just being a dry boring "engineer". After all, we are not building planes and bridges, but creating "art" that just happens to be wicked useful and pay wicked well!
Even Burger Engineers are multi-skilled (Score:2)
by EmbeddedJanitor (597831) on Monday February 04, @02:39PM (#22295872)Flipping, charring, till operating, cleaning and eating the shit from customers while keeping a smile on your face. All jobs require multiple skills.Silly whining poster probably just got out of college and is used to mommy and daddy telling him he's the greatest. Now in the real world he's just another bottom-of-the-pile programmer. Life: get one!
A great deal of crap software is actually pushed out the door against the objections of the developers that created it. Ultimately it comes down to marketing and PR and not the developers in most cases as to when a particular piece of software is ready to ship. Also, as has been pointed out, people would be unwilling to underwrite the cost of a theoretically "perfect" piece of software that would never crash (barring hardware failure, or cosmic ray induced bit flipping), because given the choice between a $50 piece of software that crashes once a week, or a $9000 piece of software that crashes never, almost everyone is going to pick the $50 one and live with the occasional crash. Does that mean developers like that? No, and most of them cringe whenever anything they wrote so much as hiccups, but sometimes they're just not given the chance or the resources (or clear documentation) they need to design it properly, because the bean counters know that the $9k piece of software the developer dreams of will never sell, but that $50 one they're puttering around with now is just about the right level already.
Perfection vs. due diligence (Score:2)
by Anonymous Brave Guy (457657) on Monday February 04, @03:47PM (#22297216)Also, as has been pointed out, people would be unwilling to underwrite the cost of a theoretically "perfect" piece of software that would never crash (barring hardware failure, or cosmic ray induced bit flipping), because given the choice between a $50 piece of software that crashes once a week, or a $9000 piece of software that crashes never, almost everyone is going to pick the $50 one and live with the occasional crash.I really hate it when these discussions become black and white. Software quality is not a binary value. It is a sliding scale with diminishing returns for effort put in, on which we are for the most part still at the "dirt cheap" end.
I doubt I would want to pay the price of near-perfection. I'll leave that for the nuclear reactors, medical facilities and space shuttles. But the cost of due diligence — which I'll assume to mean taking reasonable, well-established, tried-and-tested steps to ensure quality in this context — is not the factor of 180 you gave. It's probably not even a factor of 5, and that's today when it's a relative overhead compared to those who don't bother.
What it would mean is having to actually follow reasonable development processes that worked. No more buzzword kool-aid for you, Mr Engineer! It would mean hiring competent people as senior technical staff instead of promoting substandard but slightly cheaper code monkeys, and spending the time and money to train those working under these senior staff properly. It would mean not letting sales and marketing staff dictate the schedules at the expense of even basic quality control.
Of course, if everyone were doing this and the industry as a whole grew up, this wouldn't cost much at all, because those same good practices actually make software development more efficient. It's just that short-sighted managers with their eye on quarterly reports and personal bonuses have an active incentive not to make the long-term investments necessary to reap those long-term benefits.
The Massachusetts Institute of Technology OpenCourseWare effort has been offering free lecture notes, exams, and other resources from more than 1800 courses per its website. Some of their courses offer a substantial amount of video and audio content. I remember stumbling across this resource via my employer's intranet about a year ago. Frankly speaking, I didn't think the concept would go very far because you couldn’t earn credit…
Well, I was wrong. It’s catching fire and over 100 universities worldwide have setup similar models and some are top tier schools such as Johns Hopkins and Tufts.
I was searching for a good UNIX course but I haven't found one yet. Surprisingly, it appears MIT’s Linear Algebra course is quite popular with the OpenCourseWare community.
By the way, I don't have any affiliation with OCW or any of the higher learning institutions mentioned.
Added later:
UC Irvine OCW
Notre Dame OCW
Utah State OCW
Osaka OCW
Japan OCW Consortium
How often have you found yourself arguing with another person, and they just don't seem to understand you? Chances are they feel that you just don't understand them. You've fallen into the abstract tar pit.
Abstract discussions are like abstract art--they can be very appealing, in part because you can interpret the abstract art however you want to. People love to see what they want to see. But when it comes to technical discussions, abstract discussions are dangerous. There is a good chance someone listening to your abstract arguments will understand completely--but it won't be what you're trying to convey. To understand the abstract, they're likely creating concrete examples in their head and then arguing against your ideas based on these "private" concrete examples. The problem is, if these concrete examples aren't shared, you'll get an argument about completely different examples and understandings.
I recently worked on a 5-week project where this was really clear. There was a small group who had an idea they were trying to sell internally to get funding. Everyone else was feeling confused. Just when they thought they understood these ideas, another concept came along that contradicted what they thought they understood.
So we started a project using Expression Blend to create a "movie" of the idea. The first week we brainstormed a lot, and then drew sketches by hand of what the different screens would look like. We then presented these hand-drawn screens to a customer advisory board so we could get their feedback and help us decide what we should focus on during the next week. We intentionally used hand-drawn sketches in our discussions with customers so they wouldn't get bogged down in the small details and would just focus on the big picture.
About half way through the project we started to create actual screen mockups and animate them with Microsoft Expression Blend so it would look like a screen capture movie of an actual program--but it was all smoke and mirrors.
During the project, the team that had come up with the ideas were constantly arguing with us and saying we were asking the wrong questions. But when we had the final "movie" and showed it to them, an interesting thing happened. The conversations changed from being abstract to concrete. The idea team started to explain the details that we got wrong. And in the process, we discovered that we had gotten most of their vision correct--we just differed in some of the details.
What's more, other people who had been confused completely got the idea after seeing the movie. And again, the discussions were at a concrete level, so the discussions that came after seeing the movie were far more productive.
Robert Martin has a post about how 10% of programmers write 90% of the code. I think this is more-or-less accurate, but he seems to think that whether a programmer is a member of the elite or not is an innate quality -- that there are good programmers and poor programmers, and nobody ever moves between the two groups.I have worked on projects where I've been in the elite, and I've worked on projects where I've been in the middle, and on occasion I even qualify as a Waste Of Space for a month or two. There are several factors that influence how productive I am, personally.
First, the fewer developers in the group, the better. This is more than just being a big fish in a little pond, it's about feeling responsible for the code. If I'm in a group of 20, my contribution doesn't matter as much as if I'm in a group of four, so I don't care as much.
Second, distractions must be minimized. I enjoy helping people and answering questions, but they really cut into my concentration. Unfortunately, it's rude to ask people to use email instead of popping over for a visit or sending an instant message. Also, if I'm in an environment where I have meetings every day, scheduled such that they break my time up into hour-long chunks, then my attention is guaranteed to wander. For this reason, I tend to work best at night.
Third, history and familiarity with the code is very important. In code I've written and/or rewritten, I'm extremely productive. In code that I'm unfamiliar with, I'm not. It also helps a lot if the person who did write the code is willing to take the time to answer questions, without getting irritated. I also find that different people write the same program in vastly different ways, and if you're working on a codebase that was architected very differently from the way you would have done it, it can be difficult to ever get comfortable.
Fourth, management is important. For example, I need to feel just enough time-pressure to make me pay attention, but not so much that I give up in despair. I also need to get feedback as to how my work is perceived by users (did it suck? did it rock?) otherwise my work starts to seem pointless and I lose motivation.
Fifth, I find that my productivity has ceased to improve noticeably over time. For the first two or three years it improved dramatically, but since then I seem to have plateaued. (I currently have eight years of professional programming experience.)
If you work with someone who you think is being unproductive, perhaps you should spend some time to find out why. You might find that a very small change in their work environment can lead to a large improvement in their output. Maybe they just want to know that their code is actually useful to someone. Maybe they need free snacks, so their blood sugar doesn't get too low in the afternoon. Maybe the need to work in a quieter part of the office.
Discovering and addressing these kinds of things should be 50% of what a manager does. The other 50% should be facilitating communication both within the group and with other groups.
Posted on June 6, 2003 09:02 PM
More programming articles
A++ tags, would read again (Score:5, Funny)