In the Internet age, does software have value? Of course software is valuable in the sense that it provides service and is useful, but does software have monetary value?
If one looks at the law of supply and demand, the fact that software, like all other forms of digital content, can be endlessly reproduced and distributed at virtually no cost negates its value because software distributed this way lacks scarcity. Digital content is simply not a scarce resource. This hasn't stopped people from trying to impose artificial scarcity on digital data through the use of digital restrictions management (DRM) and draconian imaginary property laws but these approaches have had only limited success. This is not surprising as attempting to create an artificial shortage goes against the physical nature of the Internet and of computers themselves.
The Proprietary Model
If you have ever checked out my resume, you know that I spent the greater portion of my career in the proprietary software world and was, at one time, a big supporter of proprietary software. I was fortunate to have spent all of my years in the software industry working for small companies where one could wander the halls and learn every aspect of the business. In addition to being a technical manager, I had number of marketing and sales assignments as well.
Software development in the proprietary world is speculative. Typically, a product manager or marketing director is given the assignment of coming up with the "next big thing," a product that can sold to many customers at a profit. The reason that it has to be big is because proprietary software development is fantastically expensive. The product manager will present ideas to management and get approval for some personnel and a budget based on the product manager's forecasts for delivery dates and sales targets. After approval, the software development process begins, In some companies this process is very formal including requirements specifications, design reviews, test plans, etc. At the end of the process, the software product goes to market. This involves a number significant expenses including marketing, advertising, trade shows, etc.
It is important to remember that proprietary software companies don't actually sell software. They sell licenses. It is through this mechanism that they attempt to create a scarcity that gives their product value.
Proprietary software only has value once it is written. You will sometimes see product announcements appear for non-existent yet-to-be-developed products. Such products are derisively known as "vaporware" in the industry because proprietary software does not have value until it is written and actually availabile in short supply.
The Free Software Model
To members of the proprietary software community, the notion of free software appears insane. This is because they think that free software means that they have to go through all of the steps and expense of the process above and then not collect any revenue on the back-end. There are a number of problems with this assumption.
The development process for free software is fundamentally different. First off, it is not speculative. Developers of free software typically have an interest in actually using the program they want to write. It also means that free software developers are usually subject matter experts for their chosen program.
Free software is much less expensive to produce than proprietary software. The development process is much less formal than closed proprietary processes owing to the fact that development is done in the open. This allows a more natural and organic method of solving problems and fixing bugs and, unlike proprietary development, the development tools and shared software components are free. Free software also does not incur the engineering overhead of implementing "copy protection," user registration systems, and tiered product versions that are used to establish upgrade paths for proprietary products.
Finally, free software products don't have the marketing and sales expenses of proprietary software.
Making Money
While the proprietary software appears to make a lot of money now, is it sustainable? Will the Internet and its ability to perform infinite duplication and distribution drain the value from software? Only time will tell, but I'm betting that the Internet will emerge victorious. We can already see the signs of this victory with the rise of "cloud computing" which is eliminating the need for software all together. But cloud computing raises a number of issues including privacy and security, as well as freedom.
There has been a lot of discussion of how to make money with free software. Most of the ideas put forth involve charging for services. After all, Red Hat, a very successful software company, makes its money that way, but I want to suggest another possibility.
As we saw, proprietary software only has value after it is written and is available for license sales. The free software model assumes from the start that once a program is written it no longer has value because it is not scarce. In contrast to proprietary software, free software only has value before it is written. The absence of a desired software program is the ultimate scarcity. There exists an opportunity to exploit this fact. It's not really a new idea by any means. This is how the custom software business works. Clients want something and pay big money to get something written. What I envision is a business that somehow aligns many clients with developers so that the cost of development can be spread out among many clients.
What will such a business look like? That's an exercise I will leave to my more entrepreneurial readers.
Further Reading
Tuesday, April 27, 2010
Thursday, April 22, 2010
New Features In Bash Version 4.x - Part 4
In this final installment of our series, we will look at perhaps the most significant area of change in bash version 4.x: arrays.
Arrays In bash
Arrays were introduced in version 2 of bash about fifteen years ago. Since then, they have been on the fringes of shell programming. I, in fact, have never seen a shell script "in the wild" that used them. None of the scripts on LinuxCommand.org, for example, use arrays.
Why is this? Arrays are widely used in other programming languages. I see two reasons for this lack of popularity. First, arrays are not a traditional shell feature. The Bourne shell, which bash was designed to emulate and replace, offers no array support at all. Second, arrays in bash are limited to a single dimension, an unusual limitation given that virtually every other programming language supports multi-dimensional arrays.
Some Background
I devote a chapter in my book to bash arrays but briefly, bash supports single dimension array variables. Arrays behave like a column of numbers in a spread sheet. A single array variable contains multiple values called elements. Each element is accessed via an address called an index or subscript. All versions of bash starting with version 2 support integer indexes. For example, to create a five element array in bash called numbers containing the strings "zero" through "four", we would do this:
bshotts@twin7:~$ numbers=(zero one two three four)
After creating the array, we can access individual elements by specifying the array element's index:
bshotts@twin7:~$ echo ${numbers[2]}
two
The braces are required to prevent the shell from misinterpreting the brackets as wildcard characters used by pathname expansion.
Arrays are not very useful on the command line but are very useful in programming because they work well with loops. Here's an example using the array we just created:
#!/bin/bash
# array-1: print the contents of an array
numbers=(zero one two three four)
for i in {0..4}; do
echo ${numbers[$i]}
done
When executed, the script prints each element in the numbers array:
bshotts@twin7:~$ array-1
zero
one
two
three
four
mapfile Command
bash version 4 added the mapfile command. This command copies a file line-by-line into an array. It is basically a substitute for the following code:
while read line
array[i]="$line"
i=$((i + 1))
done < file
with mapfile, you can use the following in place of the code above:
mapfile array < file
mapfile handles the case of a missing newline at the end of the file and creates empty array elements when it encounters a blank line in the file. It also supports ranges within the file and the array.
Associative Arrays
By far, the most significant new feature in bash 4.x is the addition of associative arrays. Associative arrays use strings rather than integers as array indexes. This capability allow interesting new approaches to managing data. For example, could create an array called colors and use color names as indexes:
colors["red"]="#ff0000"
colors["green"]="#00ff00"
colors["blue"]="#0000ff"
Associative array elements are accessed in much the same way as integer indexed arrays:
echo ${colors["blue"]}
In the script that follows, we will look at several programming techniques that can be employed in conjunction with associative arrays. This script, called array-2, when given the name of a directory, prints a lsting of the files in the directory along with the names of the the file's owner and group owner. At the end of listing, the script prints a tally of the number of files belonging to each owner and group. Here we see the results (truncated for brevity) when the script is given the directory /usr/bin:
bshotts@twin7:~$ array-2 /usr/bin
/usr/bin/2to3-2.6 root root
/usr/bin/2to3 root root
/usr/bin/a2p root root
/usr/bin/abrowser root root
/usr/bin/aconnect root root
/usr/bin/acpi_fakekey root root
/usr/bin/acpi_listen root root
/usr/bin/add-apt-repository root root
.
.
.
/usr/bin/zipgrep root root
/usr/bin/zipinfo root root
/usr/bin/zipnote root root
/usr/bin/zip root root
/usr/bin/zipsplit root root
/usr/bin/zjsdecode root root
/usr/bin/zsoelim root root
File owners:
daemon : 1 file(s)
root : 1394 file(s)
File group owners:
crontab : 1 file(s)
daemon : 1 file(s)
lpadmin : 1 file(s)
mail : 4 file(s)
mlocate : 1 file(s)
root : 1380 file(s)
shadow : 2 file(s)
ssh : 1 file(s)
tty : 2 file(s)
utmp : 2 file(s)
Here is a listing of the script:
1 #!/bin/bash
2
3 # array-2: Use arrays to tally file owners
4
5 declare -A files file_group file_owner groups owners
6
7 if [[ ! -d "$1" ]]; then
8 echo "Usage: array-2 dir" >&2
9 exit 1
10 fi
11
12 for i in "$1"/*; do
13 owner=$(stat -c %U "$i")
14 group=$(stat -c %G "$i")
15 files["$i"]="$i"
16 file_owner["$i"]=$owner
17 file_group["$i"]=$group
18 ((++owners[$owner]))
19 ((++groups[$group]))
20 done
21
22 # List the collected files
23 { for i in "${files[@]}"; do
24 printf "%-40s %-10s %-10s\n" \
25 "$i" ${file_owner["$i"]} ${file_group["$i"]}
26 done } | sort
27 echo
28
29 # List owners
30 echo "File owners:"
31 { for i in "${!owners[@]}"; do
32 printf "%-10s: %5d file(s)\n" "$i" ${owners["$i"]}
33 done } | sort
34 echo
35
36 # List groups
37 echo "File group owners:"
38 { for i in "${!groups[@]}"; do
39 printf "%-10s: %5d file(s)\n" "$i" ${groups["$i"]}
40 done } | sort
Line 5: Unlike integer indexed arrays, which are created by merely referencing them, associative arrays must be created with the declare command using the new -A option. In this script we create five arrays as follows:
Lines 7-10: Checks to see that a valid directory name was passed as a positional parameter. If not, a usage message is displayed and the script exits with an exit status of 1.
Lines 12-20: Loop through the files in the directory. Using the stat command, lines 13 and 14 extract the names of the file owner and group owner and assign the values to their respective arrays (lines 16, 17) using the name of the file as the array index. Likewise the file name itself is assigned to the files array (line 15).
Lines 18-19: The total number of files belonging to the file owner and group owner are incremented by one.
Lines 22-27: The list of files is output. This is done using the "${array[@]}" parameter expansion which expands into the entire list of array element with each element treated as a separate word. This allows for the possibility that a file name may contain embedded spaces. Also note that the entire loop is enclosed in braces thus forming a group command. This permits the entire output of the loop to be piped into the sort command. This is necessary because the expansion of the array elements is not sorted.
Lines 29-40: These two loops are similar to the file list loop except that they use the "${!array[@]}" expansion which expands into the list of array indexes rather than the list of array elements.
Further Reading
The Linux Command Line
A Wikipedia article on associative arrays:
The Complete HTML Color Chart:
Other installments in this series: 1 2 3 4
Arrays In bash
Arrays were introduced in version 2 of bash about fifteen years ago. Since then, they have been on the fringes of shell programming. I, in fact, have never seen a shell script "in the wild" that used them. None of the scripts on LinuxCommand.org, for example, use arrays.
Why is this? Arrays are widely used in other programming languages. I see two reasons for this lack of popularity. First, arrays are not a traditional shell feature. The Bourne shell, which bash was designed to emulate and replace, offers no array support at all. Second, arrays in bash are limited to a single dimension, an unusual limitation given that virtually every other programming language supports multi-dimensional arrays.
Some Background
I devote a chapter in my book to bash arrays but briefly, bash supports single dimension array variables. Arrays behave like a column of numbers in a spread sheet. A single array variable contains multiple values called elements. Each element is accessed via an address called an index or subscript. All versions of bash starting with version 2 support integer indexes. For example, to create a five element array in bash called numbers containing the strings "zero" through "four", we would do this:
bshotts@twin7:~$ numbers=(zero one two three four)
After creating the array, we can access individual elements by specifying the array element's index:
bshotts@twin7:~$ echo ${numbers[2]}
two
The braces are required to prevent the shell from misinterpreting the brackets as wildcard characters used by pathname expansion.
Arrays are not very useful on the command line but are very useful in programming because they work well with loops. Here's an example using the array we just created:
#!/bin/bash
# array-1: print the contents of an array
numbers=(zero one two three four)
for i in {0..4}; do
echo ${numbers[$i]}
done
When executed, the script prints each element in the numbers array:
bshotts@twin7:~$ array-1
zero
one
two
three
four
mapfile Command
bash version 4 added the mapfile command. This command copies a file line-by-line into an array. It is basically a substitute for the following code:
while read line
array[i]="$line"
i=$((i + 1))
done < file
with mapfile, you can use the following in place of the code above:
mapfile array < file
mapfile handles the case of a missing newline at the end of the file and creates empty array elements when it encounters a blank line in the file. It also supports ranges within the file and the array.
Associative Arrays
By far, the most significant new feature in bash 4.x is the addition of associative arrays. Associative arrays use strings rather than integers as array indexes. This capability allow interesting new approaches to managing data. For example, could create an array called colors and use color names as indexes:
colors["red"]="#ff0000"
colors["green"]="#00ff00"
colors["blue"]="#0000ff"
Associative array elements are accessed in much the same way as integer indexed arrays:
echo ${colors["blue"]}
In the script that follows, we will look at several programming techniques that can be employed in conjunction with associative arrays. This script, called array-2, when given the name of a directory, prints a lsting of the files in the directory along with the names of the the file's owner and group owner. At the end of listing, the script prints a tally of the number of files belonging to each owner and group. Here we see the results (truncated for brevity) when the script is given the directory /usr/bin:
bshotts@twin7:~$ array-2 /usr/bin
/usr/bin/2to3-2.6 root root
/usr/bin/2to3 root root
/usr/bin/a2p root root
/usr/bin/abrowser root root
/usr/bin/aconnect root root
/usr/bin/acpi_fakekey root root
/usr/bin/acpi_listen root root
/usr/bin/add-apt-repository root root
.
.
.
/usr/bin/zipgrep root root
/usr/bin/zipinfo root root
/usr/bin/zipnote root root
/usr/bin/zip root root
/usr/bin/zipsplit root root
/usr/bin/zjsdecode root root
/usr/bin/zsoelim root root
File owners:
daemon : 1 file(s)
root : 1394 file(s)
File group owners:
crontab : 1 file(s)
daemon : 1 file(s)
lpadmin : 1 file(s)
mail : 4 file(s)
mlocate : 1 file(s)
root : 1380 file(s)
shadow : 2 file(s)
ssh : 1 file(s)
tty : 2 file(s)
utmp : 2 file(s)
Here is a listing of the script:
1 #!/bin/bash
2
3 # array-2: Use arrays to tally file owners
4
5 declare -A files file_group file_owner groups owners
6
7 if [[ ! -d "$1" ]]; then
8 echo "Usage: array-2 dir" >&2
9 exit 1
10 fi
11
12 for i in "$1"/*; do
13 owner=$(stat -c %U "$i")
14 group=$(stat -c %G "$i")
15 files["$i"]="$i"
16 file_owner["$i"]=$owner
17 file_group["$i"]=$group
18 ((++owners[$owner]))
19 ((++groups[$group]))
20 done
21
22 # List the collected files
23 { for i in "${files[@]}"; do
24 printf "%-40s %-10s %-10s\n" \
25 "$i" ${file_owner["$i"]} ${file_group["$i"]}
26 done } | sort
27 echo
28
29 # List owners
30 echo "File owners:"
31 { for i in "${!owners[@]}"; do
32 printf "%-10s: %5d file(s)\n" "$i" ${owners["$i"]}
33 done } | sort
34 echo
35
36 # List groups
37 echo "File group owners:"
38 { for i in "${!groups[@]}"; do
39 printf "%-10s: %5d file(s)\n" "$i" ${groups["$i"]}
40 done } | sort
Line 5: Unlike integer indexed arrays, which are created by merely referencing them, associative arrays must be created with the declare command using the new -A option. In this script we create five arrays as follows:
- files contains the names of the files in the directory, indexed by file name
- file_group contains the group owner of each file, indexed by file name
- file_owner contains the owner of each file, indexed by file name
- groups contains the number of files belonging to the indexed group
- owners contains the number of files belonging to the indexed owner
Lines 7-10: Checks to see that a valid directory name was passed as a positional parameter. If not, a usage message is displayed and the script exits with an exit status of 1.
Lines 12-20: Loop through the files in the directory. Using the stat command, lines 13 and 14 extract the names of the file owner and group owner and assign the values to their respective arrays (lines 16, 17) using the name of the file as the array index. Likewise the file name itself is assigned to the files array (line 15).
Lines 18-19: The total number of files belonging to the file owner and group owner are incremented by one.
Lines 22-27: The list of files is output. This is done using the "${array[@]}" parameter expansion which expands into the entire list of array element with each element treated as a separate word. This allows for the possibility that a file name may contain embedded spaces. Also note that the entire loop is enclosed in braces thus forming a group command. This permits the entire output of the loop to be piped into the sort command. This is necessary because the expansion of the array elements is not sorted.
Lines 29-40: These two loops are similar to the file list loop except that they use the "${!array[@]}" expansion which expands into the list of array indexes rather than the list of array elements.
Further Reading
The Linux Command Line
- Chapter 36 (Arrays)
- Chapter 37 (Group commands)
A Wikipedia article on associative arrays:
The Complete HTML Color Chart:
Other installments in this series: 1 2 3 4
Ubuntu 10.04 RC Has Been Released
For those of you following along with my Getting Ready For Ubuntu 10.04 series, the Release Candidate has just come out. LWN has the release announcement.
Wednesday, April 21, 2010
Hitler, as Downfall producer, orders a DMCA takedown
As you may have heard, the producers of the movie "Downfall" recently staged a DMCA takedown of all of the bunker scene parodies on YouTube, including the one I posted on this blog. Brad Templeton, well-known activist, has posted a response. You can view it here.
Further Reading
Further Reading
Thursday, April 15, 2010
stat
I was going to write the next installment in my New Features In Bash Version 4.x series today, but in thinking about the examples I want to use, I thought I should talk about the stat command first.
We're all familiar with ls. It's the first command that most people learn. Using ls you can get a lot of information about a file:
bshotts@twin7:~$ ls -l .bashrc
-rw-r--r-- 1 bshotts bshotts 3800 2010-03-25 13:18 .bashrc
Very handy. But there is one problem with ls; it's output is not very script friendly. Commands like cut cannot easily separate the fields (though awk can, but we're not talking about that yet). Wouldn't it be great if there was a command that let you get file information in a more flexible way?
Fortunately there is such a command. It's called stat. The name "stat" derives from the word status. The stat command shows the status of a file or file system. In it's basic form, it works like this:
bshotts@twin7:~$ stat .bashrc
File: `.bashrc'
Size: 3800 Blocks: 8 IO Block: 4096 regular file
Device: 801h/2049d Inode: 524890 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 1000/ bshotts) Gid: ( 1000/ bshotts)
Access: 2010-04-15 08:46:22.292601436 -0400
Modify: 2010-03-25 13:18:09.621972000 -0400
Change: 2010-03-27 08:41:31.024116233 -0400
As we can see, when given the name of a file (more than one may be specified), stat displays everything the system knows about the file short of examining its contents. We see the file name, its size including the number of blocks it's using and the size of the blocks used on the device. The attribute information includes the owner and group IDs, and the permission attributes in both symbolic and octal format. Finally we see the access (when the file was last read), modify (when the file was last written), and change (when the file attributes were last changed) times for the file.
Using the -f option, we can examine file systems as well:
bshotts@twin7:~$ stat -f /
File: "/"
ID: 9e38fe0b56e0096d Namelen: 255 Type: ext2/ext3
Block size: 4096 Fundamental block size: 4096
Blocks: Total: 18429754 Free: 10441154 Available: 9504962
Inodes: Total: 4685824 Free: 4401092
Clearly stat delivers the goods when it comes to file information, but what about that output format? I can't think of anything worse to deal with from a script writer's point-of-view (actually I can, but let's not go there!).
Here's where the beauty of stat starts to shine through. The output is completely customizable. stat supports printf-like format specifiers. Here is an example extracting just the name, size, and octal file permissions:
bshotts@twin7:~$ stat -c "%n %s %a" .bashrc
.bashrc 3800 644
The -c option provides basic formatting capabilities, while the --printf option can do even more by interpreting backslash escape sequences:
bshotts@twin7:~$ stat --printf="%n\t%s\t%a\n" .bashrc
.bashrc 3800 644
Using this format, we can produce tab-delimited output, perfect for processing by the cut command. Each of the fields in the stat output is available for formatting. See the stat man page for the complete list.
Further Reading
We're all familiar with ls. It's the first command that most people learn. Using ls you can get a lot of information about a file:
bshotts@twin7:~$ ls -l .bashrc
-rw-r--r-- 1 bshotts bshotts 3800 2010-03-25 13:18 .bashrc
Very handy. But there is one problem with ls; it's output is not very script friendly. Commands like cut cannot easily separate the fields (though awk can, but we're not talking about that yet). Wouldn't it be great if there was a command that let you get file information in a more flexible way?
Fortunately there is such a command. It's called stat. The name "stat" derives from the word status. The stat command shows the status of a file or file system. In it's basic form, it works like this:
bshotts@twin7:~$ stat .bashrc
File: `.bashrc'
Size: 3800 Blocks: 8 IO Block: 4096 regular file
Device: 801h/2049d Inode: 524890 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 1000/ bshotts) Gid: ( 1000/ bshotts)
Access: 2010-04-15 08:46:22.292601436 -0400
Modify: 2010-03-25 13:18:09.621972000 -0400
Change: 2010-03-27 08:41:31.024116233 -0400
As we can see, when given the name of a file (more than one may be specified), stat displays everything the system knows about the file short of examining its contents. We see the file name, its size including the number of blocks it's using and the size of the blocks used on the device. The attribute information includes the owner and group IDs, and the permission attributes in both symbolic and octal format. Finally we see the access (when the file was last read), modify (when the file was last written), and change (when the file attributes were last changed) times for the file.
Using the -f option, we can examine file systems as well:
bshotts@twin7:~$ stat -f /
File: "/"
ID: 9e38fe0b56e0096d Namelen: 255 Type: ext2/ext3
Block size: 4096 Fundamental block size: 4096
Blocks: Total: 18429754 Free: 10441154 Available: 9504962
Inodes: Total: 4685824 Free: 4401092
Clearly stat delivers the goods when it comes to file information, but what about that output format? I can't think of anything worse to deal with from a script writer's point-of-view (actually I can, but let's not go there!).
Here's where the beauty of stat starts to shine through. The output is completely customizable. stat supports printf-like format specifiers. Here is an example extracting just the name, size, and octal file permissions:
bshotts@twin7:~$ stat -c "%n %s %a" .bashrc
.bashrc 3800 644
The -c option provides basic formatting capabilities, while the --printf option can do even more by interpreting backslash escape sequences:
bshotts@twin7:~$ stat --printf="%n\t%s\t%a\n" .bashrc
.bashrc 3800 644
Using this format, we can produce tab-delimited output, perfect for processing by the cut command. Each of the fields in the stat output is available for formatting. See the stat man page for the complete list.
Further Reading
- The stat man page
- Chapter 10 (file attributes and permissions)
- Chapter 21 (cut command)
- Chapter 22 (printf command)
Subscribe to:
Posts (Atom)
Posts
