Bookmark this on Delicious
Recommend to StumbleUpon
This page is "browser friendly". Make your browser window as wide as you want it. The text will flow nicely for you. It is easier to read in a narrow window. With most browsers, pressing plus, minus or zero while the control key (ctrl) is held down will change the texts size. (Enlarge, reduce, restore to default, respectively.) (This is more fully explained, and there's another tip, at my Power Browsing page.)
Two sorts of help with programming are not hard to find on the web. This tutorial is of a different kind.
Easy to find: a) Reference works to tell you, say, the parameters needed by AssignFile. b) Sourcecode of completed applications, or for useful subroutines.
In this essay, you can "look over my shoulder" as I move an idea from a sketch towards a finished bit of code. I hope it will help you become more skilled at getting from an idea for an application to the finished product.
Parts of this should sound familiar, if you have read many of my tutorials. This looks at important concepts. Maybe if I write about them and you read about them, yet again, some aspect that you hadn't fully grasped from previous experience will finally "click" into place? I am largely self taught... and I can assure you: That is a very, very inefficient way to master these skills.
Programming is an art, a craft... not a science.
It's a bit like building a house.
The housebuilder starts with "I want to build a house."
Let's say the programmer started with "I want to make a new wordprocessor."
All of the ideas about good project development presented here have application to a wide gamut of design tasks. Even the design of this tutorial could be tackled thus.
Each starts with a "big picture". Each must then refine their idea somewhat...
House: Two stories, 2 bedrooms, kitchen, living room, dining room, attached garage.
WP: Way to enter formatted text, save work to file, re-load previously started jobs, print results.
Then each designer takes each bit of "the idea", and looks at it in detail, probably breaking it down further into sub-tasks, and sub-sub-tasks.
Consider the whole task to be represented by an oak tree. At ground level, if you put your arms around the trunk, you are hugging the whole project, be it build-a-house or write-a-wordprocessor.
As your focus rises up from the ground, the trunk breaks up into separate sub-trunks, they divide again into branches, and so on.
One sub-trunk might be "Bedroom" (to be divided higher up into "front bedroom" and "back bedroom"), or, in the programming example "print results"
You are done with the "breaking down" process with you know at every point on "the tree" you are confident that the description there is something you can understand. And that the pieces "fit" together. And that "big" ideas (like "print results") have been broken up again (and if necessary again and again...) until you are down to such small things that you know you can build (or program) "that".
There are three really tricky bits that make this an art or craft. It isn't like "What is 3+5", or even "What is the mineral composition of Pluto". There is An Answer to both of those, whether you know it or not.
In project design, there isn't An Answer to "What is the right way to break this up?"
When you have wisdom, you will "just naturally" break projects up well. The way Joshua Bell "just naturally" plays the violin. To get wisdom, you need experience. Experience is what you get when you try to do things without enough wisdom.
How to split it up. That's Tricky Bit One.
I said earlier that you need to keep working on the design until you know that you can do all the things at the tips of the branches. You'd be amazed at how many times I get 90% through a project, only to discover that I can't do one of the things I thought I could do... a thing essential to the functioning of what I am building.
And the third tricky bit, knowing that the pieces fit together, is equally treacherous. But reviewing, reviewing, reviewing your plan will pay dividends. Re-working a half-built something, to overcome a problem you hadn't noticed earlier is almost always tedious. It is also one of the prime causes of bugs entering a program. You end up breaking "b" as you set about fixing "a".
What I've just described is called "top down design". When you've ignored my next rule long enough, you will finally know how good it is:
The planning isn't nearly as much fun as the building... but the more effort you put into the planning, the designing, the better the "building", the better the actual typing in of all the program code will go.
(By the way, while not strictly within the definition of "building", unless you test your work as you go along, you will have problems and an unreliable, unsatisfactory final product. And doing testing well is another high art... but a story for another day.)
Once you have really finished a good design, it is time to start the building of your masterwork.
We created a virtual "tree" in the design stage, working from the "bottom" (broad statement of overall function, then function of components, then function of components- of- components, etc.)
During the "building" stage, more usually called "development", we work from the tips back to the trunk.
The "print results" component might be broken up into "choose printer", "print", "eject last page", "turn off printer".
During development, we might put together a little program which "faked" the preparation of something to print. We would then write something which would send it to a particular printer. Write the code to send the faked text to the printer. Write the code to eject the last page. Turn off printer.
When we had all of that working reliably, we could "fancify" the "select printer" part, to make it capable of accepting input from a user, to allow selection of different printers. Later, after other work, we will have a program capable of putting together something to be printed. At that point, we just "plug in" the bits we've been talking about, the bits to actually do the printing of "it".
That brings us to a very neat example of a critical element in all of this.
Think about the things I've been saying. The "prepare text to print" module is going to produce something. The "print it" module is going to need something to print.
Those two "somethings" have to be equivalent. You have to plan everything so that the characteristics of the "output" of one module are the same as the characteristics needed in the "input" of another module.
(It all goes back to what I said about programming being an art and a craft, not a science.)
Sometimes, of course, the output of one module is just a number, and the module which is going to use it only needs a number. In such cases it isn't hard to get things right. Try to see how in other cases, you are still passing things between modules, and the characteristics of the output have to be suitable for the module which accepts it as input. This is the key to being sure everything "fits together".
Don't be mislead by the fact that I am telling you only part of the story of the design (planning) and development (building) of an application I created. It may seem that the development of the sub-unit we are going to discuss came directly after the design for that sub-unit, in isolation, had been created. Not so. I am just skipping over the rest of the design work, and skipping over the parts of the development which happened before I could start building the sub-unit I will talk about here.
While you are reading this, try to get a feeling for the fact that the whole process is recursive, or fractal... like a hall of mirrors. The overall task was divided into sub-units. They were divided into sub-sub-units, and so on... but only up to a certain point. (When are you done sub-dividing? The Master Designer "just knows". The apprentice makes mistakes, and endures "learning experiences". You know you didn't sub-divide enough when, during the building phase, you find something doesn't "fit", or that you don't know how to do something important.)
Not only is the design phase recursive, the development phase is too. You assemble stuff into sub-sub-sub components. Then you assemble sub-sub-sub components into sub-sub components. Then you... until you reach the end.
The "input" to the sub(-sub-sub...) module I am talking about is as follows...
A computer with some folders already established. And in each of them, a sub-folder called "Older". And, maybe, some .jpgs. If there are no jpgs at the moment, some may appear in the folders from time to time, by processes outside the scope of what the current project need provide for.
A constant called kMaxArr. An array, holding strings, called sImageFolder[]. The elements of the array will hold things like "C:\Myfiles\Cam1\"... in other words they will specify folders. The folders may or may not exist. (Such details are important!) The last element of the array will not have an index greater than the number held in kMaxArr. (I will simply say "... last element won't be more than kMaxArr" in similar situations from here onward, implying "the value in kMaxArr" when I say "kMaxArr").
Besides the constant (kMaxArr), there is a variable, bMaxArrElInUse, which tells us the index of the last element of sImageFolder[] which currently has a meaningful value in it.
The environment also contains a string variable called sPathToAssmFldr. It to holds something like "C:\AnOtherPath\AssemblyFolder\".
Why do they have those names? Where to the values in the come from? What are in the .jpg files? None of these things matter to us, during the phase of (first) design, or (second) development which I am writing about today. They are "given"s. Their whys and wherefores have all been considered and ironed out, at a different stage of the design process.
What does matter to us is what we have to do with what we are given, and in the sub-module we are working on.
In (a rather full) nutshell...
By the end of the module, we need any jpeg files which were in any of the folders specified by sImageFolder[0] to sImageFolder[bMaxArrElInUse] to have been moved to the /Older sub-folder in each of the parent folders. Except the "youngest" jpg present. That not only stays (for now) in the parent folder, but also a copy of it has to be made to a sub-folder (name: in sSubFolder (e.g. Images\)) of sPathToAssmFldr (e.g. C:\Misc\PicHarvest\), under the name determined by the contents of the string array sDestCoreFilename[] (e.g. Img01.jpg)
Whew! That's "all". Actually, it isn't asking very much. But pinning down the details takes some words.
To recap: We will look in some folders, one at a time. (It will be done with a subroutine. Which folder we do "now" will be determined by a parameter passed to the subroutine.) We will find the youngest .jpg in the folder, and make a copy of that elsewhere. All of the other jpgs in the folder will be moved to a sub-folder of the folder. That's all.
(PS: Note to myself: In fact, when I wrote the final version of the above, it merely returned the name (the core name) of the file to copy. The actual copying was done outside of the procedure.)
Having pinned down just exactly what we want done, we can now think a bit about how we are going to do it.
Remember "divide and conquer"? Well, our task is, basically...
So far, so good. But it may not be possible to find the youngest out of all the jpgs in the folder in a single pass.
So, new plan.....
<!--NOT CODE--> repeat Pick up a number of jpgs from the source folder Find youngest in them Copy all to the "\Older" sub-folder Until every jpgs has been looked at Make a copy of the youngest jpg to the sAssmFldr folder. Make a copy of the youngest jpg back to the source folder, to "seed" things for the next time this module executes.
Now... that leaves out a few details, like what name to use when making the copy, at the end. Nor does it say how we know every file has been looked at. But it is "complete", if only an overview.
My next bit of planning was done with ink in paper, 'cause for such things I can work better that way. It supplies the details of the "how" which were lacking from the overview just presented.
We will pass the following to the subroutine, in the variables indicated...
We will have a special type available to us for use in the routine:
TFilesDetails: record Time: Integer; Size: Integer; Attr: Integer; Name: TFileName;
"Time" in this context combines a date with a time. Time=0 was a long time ago, a time before any jpg I will have on my drive.
We will have a function, written by us, called SomeFields, to move the Time, Size, Attr and Name from a TSearchRecord variable to a TFileDetails variable.
Sundry local variables we will use...
(The following picture might look like it was drawn in one attempt. I can assure you that it was not. From my first attempts to this finished product took three drafts, the second in pencil with much crossing out and re-working. Going through that drafting process forced me to focus on the parts of the design I was trying to articulate. In hindsight, I jumped too far "up" the tree. I had a general idea of this part of what I was trying to do, and I jumped to writing out too many details in one place. It would have been better if I had sub-divided the already sub-divided task before I attempted this level of detail.)
Who would have thought that what we wanted to do was so complex? It was just "find the youngest file", "move the others", after all! Part of the problem was providing for all the possible scenarios, e.g. there being no *.jpg in the folder at the start of the process. Part of the problem was my intention to do this in batches, so that the process could accommodate any number of files, even though memory limits mean that we can only do a finite number in any one pass.
Make you laugh? While typing the above, and halfway though the fine copy of the original plan, I realized that it was flawed. Oh no! It would, I think, have worked. But it occurred to me that there was a better answer available...
Frustrating to have spent all that time on the first, inferior, plan. But what if I hadn't been working so carefully? I would never have realized that the DESIGN was bad. I would have been struggling... probably terribly... with developing a bad design. Not fun. Taking the time to really work on the design paid off, because a second approach came to me while I was trying to set out the bad first "answer" clearly. (The fact that it was so complex was a "red flag" which had me thinking, "there must be a simpler (always better, if functionally equivalent) way."
There is a possible special case which would have been less likely to arise, had I used the first design. (By the way: Notice that there can be two (or more) different valid answers to a given design goal.)
The special case is this: What if, while you are determining which is the youngest .jpg in the folder, a new .jpg comes in? With the second design, that is okay.... it will still be there the next time the procedure is invoked. (And the wait to have it considered for the "output" directory will not be a problem for the wider goals of the overall application.)
Besides being less complex, so likely to be developed more quickly, and less likely to have bugs, the second design will execute more rapidly. I came up with the first design because I started with the idea that I would have to process the .jpgs in batches, that I "couldn't" process them all in a single pass. Once I'd made that "decision", the problems started piling up. I solved them... but it would have been better to go around them, which was possible when I saw that I was wrong in my assumption that I "needed" to work in batches. Sigh.
(I was also employing a sort in my original scheme... and it wasn't necessary. I need to know which file is the most recent, but I don't need the sequence of the others, do I?)
Look at either of the diagrams, in a general sense. Forget the detail. What do you see? Those lines that enclose things, and enclose enclosed things, and enclose enclosed enclosed... things are just diagrammatic representations of what I have been talking about. They show how a problem can be treated as having some general modules. And then the modules can be broken up. And the pieces broken up even more finely, etc.
Your program code, if it is well written in a modern language without GOTOs... even met those?... will "look" something like the diagrams. You can "compose" diagrams like the above on a keyboard with a wordprocessor. But I still commend "pencil and paper" to you for roughing out your plan.
So. A somewhat unusual essay for my collection of tutorials. This didn't really have "the answer" to anything specific. But I hope it will have reinforced messages which are present to varying degrees, implicitly and explicitly in many of my tutorials.
Learning to program is more about learning a process, a craft, than it is about knowing the right word to fetch the computer's idea of the current date.
If you read this essay with a sincere intention of "learning something", I don't think you'll quite be able to say what you learned. But I hope that in the weeks ahead, as you do your own programming, maybe you will do things you might not have done. I hope that you will get from an idea or requirement to a finished, working application more quickly and easily. And if not more quickly or easily, that at least the application will be more robust, more user friendly, more flexible than the things you were writing previously.
Thank you for your effort in getting through this. I assure you: It wasn't all that easy to write, either! If no one reads these things, my efforts have been wasted, and I'd hate to think that is the case.
|
|
If you visit 1&1's site from here, it helps me. They host my website, and I wouldn't put this link up for them if I wasn't happy with their service. They offer things for the beginner and the corporation.
Page tested for compliance with INDUSTRY (not MS-only) standards, using the free, publicly accessible validator at validator.w3.org. Mostly passes. There were two "unknown attributes" in Google+ button code. Sigh.
....... P a g e . . . E n d s .....