Perl Tips and Techniques

This Perl Tips blog has been permanently moved to the new WebGuru Web Programming Blog, which also covers PHP, XML, mySQL, CSS, blogging platforms and more. Here is a list of upcoming Perl tutorials.

The other day, I noticed that a couple of recent visitors to this blog were searching for a Perl-based blogging client. Rael Dornfest, Chief Technology Officer at publisher O'Reilly Media, has had a hand in a number of XML-based technologies including, I believe, an earlier version of RSS that was RDF-based. He is the "owner and trainer" of an open-source Perl-based blogging client called Blosxom (pronounced "blossom").

Blosxom appears to have a huge list of plugins and, for the most part, appears to be relatively simple (a single CGI script to upload to your web server). The web pages boast a 15-minute setup time. But with the lack of sleep that I'm getting, and despite my decade of Perl programming, I couldn't get a test post to display. However, don't let that stop you from trying. My experience with Perl says that this is simply due to some small typing error somewhere that I've made. In fact, numerous blogsites are running blosxom, and my guess is that not all of their owners are Perl programmers. (The Blosxom website has a Google pagerank of 8/10. While this isn't necessarily proof of anything, I'm thinking that there are enough people that like Blosxom to link to the site.)

If you know enough about Perl to change a couple of strings to describe your blog title, site, and URL in a text editor, then you probably know to use Blosxom. If you are a multi-blogger like myself, and are searching for a single-install platform that not only allows multiple blogs but multiple authors, then Blosxom might be for you. I will try to figure out my installation, and once I've tried out a few plugins, I'll blog some more about Blosxom.

Note: I'm still working madly trying to set up the new home for this blog, but I've been delayed. I'll post an entry here once I've finished. Thanks for your patience. For the time being, I'll continue to post here occasionally about Perl.

(c) Copyright: 2005-present, Raj Kumar Dash, http://perl-tips.blogspot.com/

Technorati : blogging platform, blosxom, perl

This blog is about to be incorporated into a single blog called "WebGuru" that will be available at my new geekSchool/MathGurus Online website (http://www.mathgurusonline.com).

The WebGuru blog will contain posts about webmastering, web programming, and website analysis in general. This includes tips and techniques for Perl, PHP, XML, CSS, mySQL, javascript, data mining/ net metrics/ web analytics, geo-plotting, RSS and more. As such, most of my technical blogs (listed in the right-hand column) are being merged into WebGuru. [Some blogs are moving elsewhere, and will be announced later.]

WebGuru will contain both "prerequisite" and "problem-solving" topics. I do not like the terms "beginner" and "advanced" for categorizing programming tips, as they sometimes scare away people who are unsure. My categorization has nothing to do with your age or your programming skills, just your knowledge of a particular web topic. Basically, if you find that you don't understand one of my "problem-solving" posts, go have a look at some of the "prerequisite" posts to either refresh your memory or learn some basic skills.

For the first few weeks, there'll be an emphasis on "prerequisite" posts so that I can later get into more complex web programming. Just watch the geekSchool website (http://www.mathgurusonline.com) for a link to WebGuru. As soon as the new blog is ready, a "WebGuru" link will appear in the left-hand navigation. Please note that my older web programming/ analysis blogs will stay as is. No more posts will be made, and commenting will be shut off, but the URL will persist.

I have 60-70 posts sketched out and want to complete a few before I go live with WebGuru (hopefully later this week or early next week). For those of you that have been searching for Perl programming/scripting tips, I have over 35 Perl tips sketched out, with more in the works.

See you at geekSchool soon.

raj

Technorati : geek school, web programming, web scripting, webmaster school, webmastering tutorials

Hello everyone. I'm back from vacation. (Hey, I still worked 16-20 hours a day on my blogs and websites, so it wasn't really a vacation). I apologize for the batch posting of this message to all of my blogs, but I'm still madly reorganizing my blogs and this is the fastest way for me to communicate with readers... (The most current links to most of my blogs and website projects can always be found at my main website, http://www.chameleonintegration.com/.)

This is a somewhat lengthy post, but if you read any of my blogs with any frequency, my recommendation is that you read it. Otherwise, just visit keep visiting the blog(s) you're interested in :D.

I have several new websites, including a social awareness site, that I launched during the last two weeks. Some of them are still being tweaked (design and architecture). I'm also in the processing of moving some blogs, amalgamating other blogs, and creating a few new ones. I have nearly 200 blog posts sketched out across all of my blogs, but not all of these posts are in publishable format. So I do have tons of content planned, including some free ebooks, tutorials, and more. I'm just one person doing all of this, so please bear with me while I'm reorganizing.

By the way, I do try to check what people are searching for and then try to write a post relating to such topics (if I don't already have some such posts). I don't consider myself a blog network per se. I'll be straight out honest and say that I want to provide free information about several topics (food, technology, entertainment, and more), and then hope that (legitimate) ad revenue supports my writing and blogging habit. I'm a former print magazine publisher and editor, so blogs are my transition into the digital realm. My experience as a former search engine webmaster and as a programmer rounds my skills out. So blogging and websites are my ideal way to spend the day. So I'm making it my business to write about what you are looking for information on, provided it falls within my areas of interest or expertise. That said, there are a few blogs on my books that I'll be collaborating on with others, including family members, friends, and acquaintances.

So the scope of the "Chameleon Integration Systems" (CIS) blogs is expanding. I just have to keep it manageable so I can increase quality. The blog page templates I'm using will be changing on many of my blogs as I changing blogging platforms. For those that are curious, I currently use Blogger.com, WordPress and MovableType. I'll be trying out Mambo, bMachine, and others as well. Why all the platforms? Well, I have close to a decade of experience evaluating very high end ($500,000-$2,000,000) CMSes (Content Management Systems) for many large companies. Now I'm focusing on OpenSource solutions, specifically on software that can help bloggers set up both blogs and regular websites, plus online shopping. My "Chameleon Integration" motto is "Making the Internet Easy". So I'll be writing about my findings, for those that are interested.

Finally, just a note about blog posting schedules. I will not be posting on Sundays (I live in North America, time zone -0500., same zone as New York and Toronto). Sundays will be a day that I analyze stats, design new web pages, and sketch out the next week's worth of posts, and basically unwind. While I am aiming at posting daily to most blogs, I am still doing a lot of infrastructure work, so I won't be up to speed right away. I'll be posting some entries later today, but I probably won't be posting to every blog (new and old) until next week or the next. So I'll try to keep "current events" information posted at my main website, http://www.chameleonintegration.com/. I hope you'll visit again, and drop off comments about what you'd like to see information on.

cheers,

raj kumar dash

In the last Perl-Tips post, I discussed the NCSA Extended Log Format for web servers. Please read that post before continuing with this one. The discussion here assumes the Extended Log Format for the web server access log.

Let's review the problem at hand. We have a website for which we are getting visitors. We want to do some analysis (web metrics, web analytics) for the website: who is visiting, how often, and which pages? To do this, our secondary goal has to be to transfer the information from the website's web server access log into a database. We are a few posts away from this goal. We have devised a temporary XML format, which we'll use later to transfer the access log data into a database.

To create the WSML (Web Server Markup Language) XML output file, we need to parse the access log. To do that, we need to come up with the appropriate Perl regular expressions to properly extract the fields of each entry in the access.log. Regular expressions are a sort of wildcard/pattern rule that we can specify to extract all or some "fields" in a line of data. I can't give you a full discussion of regular expressions here. (This and this are two of the best books available on the topic.) You'll have to at least under stand the basics of Perl pattern-matching before continuing. (Try checking some of the Perl perldoc documentation that should have come with your Perl distribution first.)

However, before we actually get into true regular expressions, let me show you a way to extract some of the information of the web server access log using the Perl split() function. We cannot accurately extract every field in every record, but we can extract some important ones. The rest of this post is in PDF format [176 Kb]. (Before you read the PDF file, please read the previous posts. There is also the assumption that you know enough Perl to follow along.) The post following this one will get into using regular expressions to extract all of the fields in each access log record.

(c) Copyright 2005-present, Raj Kumar Dash, http://perl-tips.blogspot.com

Technorati : access log, perl, programming, web analytics, web metrics, web script, web server