HM2K.com http://www.hm2k.com The research of an internet entrepreneur and IT consultant Thu, 25 Feb 2010 22:02:20 +0000 http://wordpress.org/?v=2.9.2 en hourly 1 Universal Currency Converter TCL for Eggdrop http://www.hm2k.com/posts/universal-currency-converter-tcl-for-eggdrop http://www.hm2k.com/posts/universal-currency-converter-tcl-for-eggdrop#comments Thu, 25 Feb 2010 14:41:31 +0000 hm2k http://www.hm2k.com/?p=602 This is a currency converter written in TCL for Eggdrop, which is similar to the ucc.mrc I wrote for mIRC.

I used to use the xeucc.tcl, but that script no longer works and hasn’t been updated since 2008.

> !ucc 9 usd gbp
<Bot> Could not obtain results from XE.com, sorry!

Further more xeucc.tcl uses the XE.com website to gather the information, however they don’t like us doing this, and have since added the following message:

WARNING: Automated extraction of rates is prohibited under the Terms of Use.

Other scripts such as Incith’s Exchange also use a website. This one uses Yahoo Finance. The problem is that this website is subject to change at any time without prior notice.

Parsing a website that is subject to change is not always a good idea. Instead what is needed is an API which is easy to parse and extract the data and also unlikely to change formatting.

Although XE.com does have an API, but it is not free. This is no good.

I have decided instead to use the Yahoo Developer Network Currency Converter API, which is much easier to parse and permits this kind of usage.

There is a choice of two URLs which you can call to get the data:

http://finance.yahoo.com/d/quotes.csv?e=.csv&f=sl1d1t1&s=USDGBP=X

http://download.finance.yahoo.com/d/?f=sl1d1t1&s=USDGBP=X

Both examples should provide you with a csv file to download which will contain the currency information.

I have used the second URL in my script.

Usage

> .ucc
<Bot> Usage: .ucc <amount> <from> <to>
> .ucc 9 usd gbp
<Bot> HM2K, * UCC: 9 USD is 5.8905 GBP as of 2/25/2010 9:22am

Download

If you like it, or even if you don’t, please leave a comment below. Thanks.

]]> This is a currency converter written in TCL for Eggdrop, which is similar to the ucc.mrc I wrote for mIRC.

I used to use the xeucc.tcl, but that script no longer works and hasn’t been updated since 2008.

> !ucc 9 usd gbp
<Bot> Could not obtain results from XE.com, sorry!

Further more xeucc.tcl uses the XE.com website to gather the information, however they don’t like us doing this, and have since added the following message:

WARNING: Automated extraction of rates is prohibited under the Terms of Use.

Other scripts such as Incith’s Exchange also use a website. This one uses Yahoo Finance. The problem is that this website is subject to change at any time without prior notice.

Parsing a website that is subject to change is not always a good idea. Instead what is needed is an API which is easy to parse and extract the data and also unlikely to change formatting.

Although XE.com does have an API, but it is not free. This is no good.

I have decided instead to use the Yahoo Developer Network Currency Converter API, which is much easier to parse and permits this kind of usage.

There is a choice of two URLs which you can call to get the data:

http://finance.yahoo.com/d/quotes.csv?e=.csv&f=sl1d1t1&s=USDGBP=X

http://download.finance.yahoo.com/d/?f=sl1d1t1&s=USDGBP=X

Both examples should provide you with a csv file to download which will contain the currency information.

I have used the second URL in my script.

Usage

> .ucc
<Bot> Usage: .ucc <amount> <from> <to>
> .ucc 9 usd gbp
<Bot> HM2K, * UCC: 9 USD is 5.8905 GBP as of 2/25/2010 9:22am

Download

If you like it, or even if you don’t, please leave a comment below. Thanks.

]]> http://www.hm2k.com/posts/universal-currency-converter-tcl-for-eggdrop/feed 0 How to convert an image to greyscale using PHP http://www.hm2k.com/posts/how-to-convert-an-image-to-greyscale-using-php http://www.hm2k.com/posts/how-to-convert-an-image-to-greyscale-using-php#comments Thu, 10 Dec 2009 14:39:30 +0000 hm2k http://www.hm2k.com/?p=583 I was trying to find a neat way to grey out some “powered by” logos for a website I’m developing.

I remembered reading about this very thing in .net magazine/august 2009 (i191,p100), which said:

“Sponsor and associate logos can be a challenge for a designer who is making a valiant attempt to create a cohesive website layout. Logos can be contrasting colour schemes and can be displayed at a wide range of different aspect ratios. When faced with this challenge, I recommend greying out the logos and having them appear in their respective colours upon mouse-over. When placing logos next to each other, I’d consider using visual weight versus actual size to determine spacing.”

- Samantha Warren @ badassideas.com

So, sure we’ve established that it’s a good idea, the question is how to achieve it…

It’s really quite simple, there’s two parts:

  1. Convert the image to greyscale using PHP.
  2. Write HTML and javascript that will change the image source on mouseover.

Before I get started, I need to point out that because I’m British, I use “Grey”, instead of the American “Gray”, apart from in the actual code.

I very quickly discovered a number of ways to convert an image from colour to greyscale, however the easiest way appeared to be using the imagefilter() function with the IMG_FILTER_GRAYSCALE greyscale filter which produced the results I was looking for. I found you can also very easily add all sorts of effects to images using PHP.

I kept it simple and wrote a function that would convert an image from jpeg, gif or png to a png greyscale image. I called it imagegray().

It’s very easy to use, you just pass the filename to the function and it will display the image in greyscale. For example:

<?php

$i=isset($_REQUEST['i'])?$_REQUEST['i']:”;
if ($i) { imagegrey($i); }

?>

Next, it’s onto the HTML, which although isn’t exactly difficult, you may stumble if you’re unsure…

<img onmouseover=”this.src=’image.jpg’” onmouseout=”this.src=’imagegrey.php?i=image.jpg’” src=”imagegrey.php?i=image.jpg” alt=”image” border=”0″>

That’s it, that’s all there is to it.

If you run a large site, for scalability reasons you may wish to involve caching, but for most small sites, this method is quick and effective.

Enjoy!

]]> I was trying to find a neat way to grey out some “powered by” logos for a website I’m developing.

I remembered reading about this very thing in .net magazine/august 2009 (i191,p100), which said:

“Sponsor and associate logos can be a challenge for a designer who is making a valiant attempt to create a cohesive website layout. Logos can be contrasting colour schemes and can be displayed at a wide range of different aspect ratios. When faced with this challenge, I recommend greying out the logos and having them appear in their respective colours upon mouse-over. When placing logos next to each other, I’d consider using visual weight versus actual size to determine spacing.”

- Samantha Warren @ badassideas.com

So, sure we’ve established that it’s a good idea, the question is how to achieve it…

It’s really quite simple, there’s two parts:

  1. Convert the image to greyscale using PHP.
  2. Write HTML and javascript that will change the image source on mouseover.

Before I get started, I need to point out that because I’m British, I use “Grey”, instead of the American “Gray”, apart from in the actual code.

I very quickly discovered a number of ways to convert an image from colour to greyscale, however the easiest way appeared to be using the imagefilter() function with the IMG_FILTER_GRAYSCALE greyscale filter which produced the results I was looking for. I found you can also very easily add all sorts of effects to images using PHP.

I kept it simple and wrote a function that would convert an image from jpeg, gif or png to a png greyscale image. I called it imagegray().

It’s very easy to use, you just pass the filename to the function and it will display the image in greyscale. For example:

<?php

$i=isset($_REQUEST['i'])?$_REQUEST['i']:”;
if ($i) { imagegrey($i); }

?>

Next, it’s onto the HTML, which although isn’t exactly difficult, you may stumble if you’re unsure…

<img onmouseover=”this.src=’image.jpg’” onmouseout=”this.src=’imagegrey.php?i=image.jpg’” src=”imagegrey.php?i=image.jpg” alt=”image” border=”0″>

That’s it, that’s all there is to it.

If you run a large site, for scalability reasons you may wish to involve caching, but for most small sites, this method is quick and effective.

Enjoy!

]]> http://www.hm2k.com/posts/how-to-convert-an-image-to-greyscale-using-php/feed 1 PHP OpenSRS Client http://www.hm2k.com/posts/php-opensrs-client http://www.hm2k.com/posts/php-opensrs-client#comments Wed, 25 Nov 2009 13:42:50 +0000 hm2k http://www.hm2k.com/?p=579 This is only a quick message to say that I am now developing the PHP OpenSRS Client project.

OpenSRS-PHP is a PHP-based client class that communicates with the TuCows OpenSRS Domain Name Registry System via an API.

I have just released version 2.8.1 with many long awaited fixes.

Enjoy!

]]> This is only a quick message to say that I am now developing the PHP OpenSRS Client project.

OpenSRS-PHP is a PHP-based client class that communicates with the TuCows OpenSRS Domain Name Registry System via an API.

I have just released version 2.8.1 with many long awaited fixes.

Enjoy!

]]> http://www.hm2k.com/posts/php-opensrs-client/feed 0 worldtime.tcl http://www.hm2k.com/posts/worldtime-tcl http://www.hm2k.com/posts/worldtime-tcl#comments Fri, 20 Nov 2009 10:04:23 +0000 hm2k http://www.hm2k.com/?p=573 Sometimes you need to know the time somewhere else in the world. I find it useful to get my eggdrop to return the time of another timezone…

> .tz london
<Bot> HM2K, The time for the london timezone is Fri Feb 13 23:31:30 2009

However, much like my old weather.tcl, I realised that my worldtime.tcl had passed it’s sell by date and had now expired.

My old worldtime.tcl which was based on a script by Murf which used worldtimeserver.com to gather it’s data had stopped working.

It would seem that worldtimeserver.com had changed their markup which meant that it was no longer possible to parse the correct data from the HTML.

Never mind, I thought, there must be a better way, that doesn’t need to use a third party website, that won’t stop working.

After all operating systems have worldtime build in without using a website, so how do they do it?

So, I did some investigation…

I discovered that on Linux based systems (including FreeBSD) the time zone information was stored locally.

/usr/share/zoneinfo/

Using a bit of Google magic, I managed to locate a few pre-made procedures to help me read the time zone information.

Using these functions I was able to find the right timezone and get the GMT offset which allowed me to work out the time in that timezone using the unix ticks.

At the moment, because it reads the time zone files that are only available on Linux based systems, it will not work on windrops.

However, what I may do next time is build a function that saves the timezone information into a database file which can be used on windows systems and will also give the Linux systems a cache file to make it respond quicker.

So now you’ll probably want to download it…

Hope it works for you.

Enjoy!

Note: If you find this useful, or have an issue with the script, please add a comment below. Thanks.

]]> Sometimes you need to know the time somewhere else in the world. I find it useful to get my eggdrop to return the time of another timezone…

> .tz london
<Bot> HM2K, The time for the london timezone is Fri Feb 13 23:31:30 2009

However, much like my old weather.tcl, I realised that my worldtime.tcl had passed it’s sell by date and had now expired.

My old worldtime.tcl which was based on a script by Murf which used worldtimeserver.com to gather it’s data had stopped working.

It would seem that worldtimeserver.com had changed their markup which meant that it was no longer possible to parse the correct data from the HTML.

Never mind, I thought, there must be a better way, that doesn’t need to use a third party website, that won’t stop working.

After all operating systems have worldtime build in without using a website, so how do they do it?

So, I did some investigation…

I discovered that on Linux based systems (including FreeBSD) the time zone information was stored locally.

/usr/share/zoneinfo/

Using a bit of Google magic, I managed to locate a few pre-made procedures to help me read the time zone information.

Using these functions I was able to find the right timezone and get the GMT offset which allowed me to work out the time in that timezone using the unix ticks.

At the moment, because it reads the time zone files that are only available on Linux based systems, it will not work on windrops.

However, what I may do next time is build a function that saves the timezone information into a database file which can be used on windows systems and will also give the Linux systems a cache file to make it respond quicker.

So now you’ll probably want to download it…

Hope it works for you.

Enjoy!

Note: If you find this useful, or have an issue with the script, please add a comment below. Thanks.

]]> http://www.hm2k.com/posts/worldtime-tcl/feed 3 Gmail needs an API http://www.hm2k.com/posts/gmail-needs-an-api http://www.hm2k.com/posts/gmail-needs-an-api#comments Fri, 13 Nov 2009 16:05:06 +0000 hm2k http://www.hm2k.com/?p=570 Yesterday I spent some time investigating Google Wave. A very interesting new product from Google. Not only that but an interesting protocol.

If you think about it, Google has mostly only used other protocols for communication in the past, now it’s creating it’s own. They clearly want to create a completely new protocol that will effectively be the future of these old primitive protocols.

For this to catch on, it will either have to run directly along side smtp and xmmp or layer on top of them.

We’re going to need a way to pass requests over http to smtp or pop3/imap, and what better way to do it than with gmail?

Sure, but there’s no API…

It’s argued that all you need to do is roll your own “api” or library class to use smtp and imap server provided by Google Mail (gmail) to send and receive emails.

The problem with this is that you’re forced to hand the app your username and password. Naturally this raises security concerns.

Of course this isn’t a big problem if you only use your own username and password and manage to store it on your server security.

The problem comes when others want to use their gmail accounts with your web app. They have not choice but to hand over their username and password. What can possibly go wrong?

  • Their username and password gives you FULL access to their account, including account actions such as changing passwords.

My proposal would be for an API that allowed you to interface with gmail, in particular to send emails (which has the biggest scope here).

To access the API you could generate an API Key from within gmail that would allow you to interface with gmail, but limited to mailing functions, rather than account settings.

An alternative to this is similar to the “GoogleCode.com Password”, used by project members when checking out or committing source code changes, or when using command-line tools to upload files to the project “Downloads” tab.

If you could set a “Google Mail SMTP Password” or similar, which would only permit you to access the gmail SMTP server, but not gmail itself, this would also allow you to overcome the security issues mentioned above.

However, this alternative wouldn’t help towards integration with Google Wave as much as an api would.

In summary, to resolve the security issue an API is an option, but for Google Wave integration too, it’s essentially a must.

I hope someone at Google reads this and agrees with me, for that matter, I hope you or anyone reading this agrees with me.

]]> Yesterday I spent some time investigating Google Wave. A very interesting new product from Google. Not only that but an interesting protocol.

If you think about it, Google has mostly only used other protocols for communication in the past, now it’s creating it’s own. They clearly want to create a completely new protocol that will effectively be the future of these old primitive protocols.

For this to catch on, it will either have to run directly along side smtp and xmmp or layer on top of them.

We’re going to need a way to pass requests over http to smtp or pop3/imap, and what better way to do it than with gmail?

Sure, but there’s no API…

It’s argued that all you need to do is roll your own “api” or library class to use smtp and imap server provided by Google Mail (gmail) to send and receive emails.

The problem with this is that you’re forced to hand the app your username and password. Naturally this raises security concerns.

Of course this isn’t a big problem if you only use your own username and password and manage to store it on your server security.

The problem comes when others want to use their gmail accounts with your web app. They have not choice but to hand over their username and password. What can possibly go wrong?

  • Their username and password gives you FULL access to their account, including account actions such as changing passwords.

My proposal would be for an API that allowed you to interface with gmail, in particular to send emails (which has the biggest scope here).

To access the API you could generate an API Key from within gmail that would allow you to interface with gmail, but limited to mailing functions, rather than account settings.

An alternative to this is similar to the “GoogleCode.com Password”, used by project members when checking out or committing source code changes, or when using command-line tools to upload files to the project “Downloads” tab.

If you could set a “Google Mail SMTP Password” or similar, which would only permit you to access the gmail SMTP server, but not gmail itself, this would also allow you to overcome the security issues mentioned above.

However, this alternative wouldn’t help towards integration with Google Wave as much as an api would.

In summary, to resolve the security issue an API is an option, but for Google Wave integration too, it’s essentially a must.

I hope someone at Google reads this and agrees with me, for that matter, I hope you or anyone reading this agrees with me.

]]> http://www.hm2k.com/posts/gmail-needs-an-api/feed 3 PlusFM Track Feed http://www.hm2k.com/posts/plusfm-track-feed http://www.hm2k.com/posts/plusfm-track-feed#comments Thu, 12 Nov 2009 15:18:53 +0000 hm2k http://www.hm2k.com/?p=567 Recently an online radio station I listen to called PlusFM stopped displaying track titles on their stream. This was a pain for quickly displaying the current track you were listening to, however I did figure out a work around…

Here’s my setup…

I have two machines in my office, one has a hifi system attached, the other does not.

I use the lightweight media player Foobar2000 (foobar2000.org), developed by Peter Pawlowski, a former freelance contractor for Nullsoft. So you know it’s a good alternative to Winamp.

In Foobar2000 I add the location for PlusFM’s stream:

http://91.121.48.90

I then use the last.fm plugin for audio scrobbling in Foobar2000 (foo_audioscrobbler.dll).

This allows me to feed the track I’m listening to the last.fm API.

I can then call their API and display the current track I’m playing on any device, anywhere in the world.

I currently use it in combination with my “last.fm” mIRC script, which calls the API, reads the currently playing track, then displays it on screen when you trigger it using the /amp command.

This mean that when I hear a song I like, I could display it to share it with others, or just display it for my own interest.

This was great, until recently, when plusFM changed, and decided to stop displaying the track names on their feed…

* +HM2K is listening to www.plusfm.net – current track available on www.plusfm.net <www.last.fm/user/hm2k>

I understand that you guys want more hits to your site, but you’re messing up my last.fm feed.

Perhaps PlusFM could sign up to last.fm and feed the audio scrobbler, then I can simply read it from there!

Or perhaps I could read it directly from your site…

So it turns out there’s a feed on the site I can read from instead…

http://www.plusfm.net/titre_plusfm.php

So I wrote a little script for it called plusfm.mrc…

[12:16:48 pm] * HM2K now playing Justice – Genesis <www.plusfm.net>

]]> Recently an online radio station I listen to called PlusFM stopped displaying track titles on their stream. This was a pain for quickly displaying the current track you were listening to, however I did figure out a work around…

Here’s my setup…

I have two machines in my office, one has a hifi system attached, the other does not.

I use the lightweight media player Foobar2000 (foobar2000.org), developed by Peter Pawlowski, a former freelance contractor for Nullsoft. So you know it’s a good alternative to Winamp.

In Foobar2000 I add the location for PlusFM’s stream:

http://91.121.48.90

I then use the last.fm plugin for audio scrobbling in Foobar2000 (foo_audioscrobbler.dll).

This allows me to feed the track I’m listening to the last.fm API.

I can then call their API and display the current track I’m playing on any device, anywhere in the world.

I currently use it in combination with my “last.fm” mIRC script, which calls the API, reads the currently playing track, then displays it on screen when you trigger it using the /amp command.

This mean that when I hear a song I like, I could display it to share it with others, or just display it for my own interest.

This was great, until recently, when plusFM changed, and decided to stop displaying the track names on their feed…

* +HM2K is listening to www.plusfm.net – current track available on www.plusfm.net <www.last.fm/user/hm2k>

I understand that you guys want more hits to your site, but you’re messing up my last.fm feed.

Perhaps PlusFM could sign up to last.fm and feed the audio scrobbler, then I can simply read it from there!

Or perhaps I could read it directly from your site…

So it turns out there’s a feed on the site I can read from instead…

http://www.plusfm.net/titre_plusfm.php

So I wrote a little script for it called plusfm.mrc…

[12:16:48 pm] * HM2K now playing Justice – Genesis <www.plusfm.net>

]]> http://www.hm2k.com/posts/plusfm-track-feed/feed 2 weather.tcl http://www.hm2k.com/posts/weather-tcl http://www.hm2k.com/posts/weather-tcl#comments Thu, 05 Nov 2009 00:37:14 +0000 hm2k http://www.hm2k.com/?p=560 Today, after I decided to hang out in #eggtcl on EFnet, someone asked the following:

<daILLeST> anyine know of a weather tcl that actually works? All the ones I find are outdated, or in another language…

I decided it was time that I updated my old weather.tcl based on a script by Ycarus for eggdrop bots.

Before, many scripts used weather underground (wunderground.com) to gather their feed.

I believe this is because there were many existing TCL based software that used this site to gather their data via telnet.

Telnet access to their server stopped working correctly, so scripts began using their http site to gather data. This too eventually stopped working as once they changed their site, the scripts could no longer parse the html for the data.

Enter Google.

Google has a weather API that they use for their “iGoogle“, which is very handy.

Although it appears to be undocumented by Google, working with the API is easy enough…

http://www.google.com/ig/api?weather=london&hl=en

The query string contains a “weather” parameter, which is generally city or postcode and the “hl” parameter which is the language.

This will return an XML formatted result for the city of “London, England”.

We can then parse the data we want from this and return it accordingly.

So, put it all together, a bit of TCL care (get it?), and we’re done…

It works well, and returns a result similar to this:

> .wz london
<Bot> HM2K, * Weather: London, England: Mostly Cloudy, 8ºC Humidity: 87% Wind: W at 9 mph

Hope you find this useful.

What I may do next time…

Since Google’s Weather API is completely officially undocumented it’s worth noting that in future it may be wise to use Yahoo’s weather API, which is fully and officially documented.

]]> Today, after I decided to hang out in #eggtcl on EFnet, someone asked the following:

<daILLeST> anyine know of a weather tcl that actually works? All the ones I find are outdated, or in another language…

I decided it was time that I updated my old weather.tcl based on a script by Ycarus for eggdrop bots.

Before, many scripts used weather underground (wunderground.com) to gather their feed.

I believe this is because there were many existing TCL based software that used this site to gather their data via telnet.

Telnet access to their server stopped working correctly, so scripts began using their http site to gather data. This too eventually stopped working as once they changed their site, the scripts could no longer parse the html for the data.

Enter Google.

Google has a weather API that they use for their “iGoogle“, which is very handy.

Although it appears to be undocumented by Google, working with the API is easy enough…

http://www.google.com/ig/api?weather=london&hl=en

The query string contains a “weather” parameter, which is generally city or postcode and the “hl” parameter which is the language.

This will return an XML formatted result for the city of “London, England”.

We can then parse the data we want from this and return it accordingly.

So, put it all together, a bit of TCL care (get it?), and we’re done…

It works well, and returns a result similar to this:

> .wz london
<Bot> HM2K, * Weather: London, England: Mostly Cloudy, 8ºC Humidity: 87% Wind: W at 9 mph

Hope you find this useful.

What I may do next time…

Since Google’s Weather API is completely officially undocumented it’s worth noting that in future it may be wise to use Yahoo’s weather API, which is fully and officially documented.

]]> http://www.hm2k.com/posts/weather-tcl/feed 3 The specified network name is no longer available samba http://www.hm2k.com/posts/the-specified-network-name-is-no-longer-available-samba http://www.hm2k.com/posts/the-specified-network-name-is-no-longer-available-samba#comments Tue, 03 Nov 2009 13:02:14 +0000 hm2k http://www.hm2k.com/?p=218 I keep getting this error:

Error Copying File or Folder

Cannot copy xxx: The specified network name is no longer available.

Why?

Well, I’m trying to copy a folder from my Windows XP SP2 Pro workstation onto my network share drive, which in fact is a samba (smb) server running on my “CentOS release 4.6 (Final)” box.

First thing to do is to check the error logs (generally) at “/var/log/samba”…

Sep  8 16:26:13 blade smbd[5913]: [2008/09/08 16:26:13, 0] lib/util_sock.c:read_data(534)
Sep  8 16:26:13 blade smbd[5913]:   read_data: read failure for 4 bytes to client 192.168.0.3. Error = Connection reset by peer
Sep  8 16:26:14 blade smbd[5914]: [2008/09/08 16:26:14, 0] lib/util_sock.c:set_socket_options(237)
Sep  8 16:26:14 blade smbd[5914]:   Unknown socket option TCP_NODELAY_SO_RCVBUF
Sep  8 16:26:14 blade smbd[5914]: [2008/09/08 16:26:14, 0] lib/util_sock.c:set_socket_options(237)
Sep  8 16:26:14 blade smbd[5914]:   Unknown socket option SO_SNBUF

Doesn’t seem very helpful…

I can’t work out why I’m getting these errors, why is the peer being reset?

I learned that the peer is the client machine that the server is peering to, which means the problem is that my workstation is disconnecting me.

I checked my Event Viewer (%SystemRoot%\system32\eventvwr.msc), which said:

NetBT failed to process a request because it encountered OutOfResources exception(s) in the last 1 hour. Event ID: 4322

I might have known that the problem would be with windows and not linux.

However, maybe I spoke too soon, as a quick search lands me on the redhat site, which states:

* when Windows® clients performed file operations on files stored on a
Samba share, various error messages popped-up, and the “Event ID: 4322″
error was logged on the Windows® clients:

NetBT failed to process a request because it encountered OutOfResources
exception(s) in the last 1 hour.

Which does describe the problem I’m having exactly, although i’ve only been experiencing this recently…

On the other hand the Microsoft site suggests I should Disable then Enable the network adapter…

I tried this, and it appeared to work… At least for now…

Low and behold I soon got this error a short time later. Now what?

I decided to take a look into the samba bug mentioned above. Apparently the problem is seen in Samba 3.0.25 and 3.0.25a, but not Samba 3.0.24. I am running 3.0.25b-1.el4_6.5 (smbd –version).

After investigating my yum log (/var/log/yum.log) I see the following…

Jul 08 01:38:04 Updated: samba-common.i386 3.0.25b-1.el4_6.5
Jul 08 01:39:11 Updated: samba.i386 3.0.25b-1.el4_6.5
Jul 08 01:39:17 Updated: samba-client.i386 3.0.25b-1.el4_6.5

This means that it’s not very likely to be that as I’ve been running it for more than two months, and i’ve only recently switched to this workstation (after an uber hardware failure).

I tried from another machine, and I can copy anything without seeing these errors… The plot thickens…

I’m now convinced that the problem is with my workstation and not the server. Now what?

I noticed a thread by a guy who mentioned that his machine (like mine) was up to date, SP2, but did not have IE7, unlike my other machines. This interested me. Maybe his bug isn’t fixed in the version I’m using? I think it is, it’s quite old…

However, it seems his solution was to upgrade to 3.0.25c and/or change the connection from “netbios-ssn” (port 139) to “microsoft-ds” (port 445)… somehow. I checked on both ends, and appears I’m already using “microsoft-ds” on this machine anyway…

Frustration is starting to kick in…

I’m out of ideas, perhaps I should roll back to an older version, see if that helps? My logs report this as been the previous version:

Jun 07 17:01:55 Updated: samba-common.i386 3.0.10-1.4E.12.2
Jun 07 17:04:29 Updated: samba-client.i386 3.0.10-1.4E.12.2
Jun 07 17:07:44 Updated: samba.i386 3.0.10-1.4E.12.2

I had second thoughts about this when reading the samba change log (rpm -q –changelog samba). There seems to be quite a few security fixes that I’d rather keep…

I’m totally out of ideas… “yum remove samba” it is… However, apparently to install an old version I have to first install a plugin called “yum-allowdowngrade”.

This is getting complicated, especially when it probably isn’t a problem with the server.

I decided to try an updated version instead…

yum remove samba
yum remove samba-common
rpm -i http://charliebrady.org/samba/samba-common-3.0.30-0.i386.rpm

rpm -i ftp://ftp.pbone.net/mirror/www.startcom.org/AS-4.0.0/os/i386/StartCom/RPMS/perl-Convert-ASN1-0.18-3.noarch.rpm
rpm -i http://charliebrady.org/samba/samba-3.0.30-0.i386.rpm
service samba start

It’s installed, and it started, however I appear to have lost my settings.

mv /etc/samba/smb.conf /etc/samba/smb.conf.recent
mv /etc/samba/smb.conf.rpmsave /etc/samba/smb.conf
service smb restart

Testing, and so far so good!

In summary, when I was convinced it was the silly windows based client workstation, instead it was the insanely overly complex linux server. This could all be avoided if redhat/centos wasn’t so keen on backporting and I had the latest version in the first place.

I look forward to the updates.

]]> I keep getting this error:

Error Copying File or Folder

Cannot copy xxx: The specified network name is no longer available.

Why?

Well, I’m trying to copy a folder from my Windows XP SP2 Pro workstation onto my network share drive, which in fact is a samba (smb) server running on my “CentOS release 4.6 (Final)” box.

First thing to do is to check the error logs (generally) at “/var/log/samba”…

Sep  8 16:26:13 blade smbd[5913]: [2008/09/08 16:26:13, 0] lib/util_sock.c:read_data(534)
Sep  8 16:26:13 blade smbd[5913]:   read_data: read failure for 4 bytes to client 192.168.0.3. Error = Connection reset by peer
Sep  8 16:26:14 blade smbd[5914]: [2008/09/08 16:26:14, 0] lib/util_sock.c:set_socket_options(237)
Sep  8 16:26:14 blade smbd[5914]:   Unknown socket option TCP_NODELAY_SO_RCVBUF
Sep  8 16:26:14 blade smbd[5914]: [2008/09/08 16:26:14, 0] lib/util_sock.c:set_socket_options(237)
Sep  8 16:26:14 blade smbd[5914]:   Unknown socket option SO_SNBUF

Doesn’t seem very helpful…

I can’t work out why I’m getting these errors, why is the peer being reset?

I learned that the peer is the client machine that the server is peering to, which means the problem is that my workstation is disconnecting me.

I checked my Event Viewer (%SystemRoot%\system32\eventvwr.msc), which said:

NetBT failed to process a request because it encountered OutOfResources exception(s) in the last 1 hour. Event ID: 4322

I might have known that the problem would be with windows and not linux.

However, maybe I spoke too soon, as a quick search lands me on the redhat site, which states:

* when Windows® clients performed file operations on files stored on a
Samba share, various error messages popped-up, and the “Event ID: 4322″
error was logged on the Windows® clients:

NetBT failed to process a request because it encountered OutOfResources
exception(s) in the last 1 hour.

Which does describe the problem I’m having exactly, although i’ve only been experiencing this recently…

On the other hand the Microsoft site suggests I should Disable then Enable the network adapter…

I tried this, and it appeared to work… At least for now…

Low and behold I soon got this error a short time later. Now what?

I decided to take a look into the samba bug mentioned above. Apparently the problem is seen in Samba 3.0.25 and 3.0.25a, but not Samba 3.0.24. I am running 3.0.25b-1.el4_6.5 (smbd –version).

After investigating my yum log (/var/log/yum.log) I see the following…

Jul 08 01:38:04 Updated: samba-common.i386 3.0.25b-1.el4_6.5
Jul 08 01:39:11 Updated: samba.i386 3.0.25b-1.el4_6.5
Jul 08 01:39:17 Updated: samba-client.i386 3.0.25b-1.el4_6.5

This means that it’s not very likely to be that as I’ve been running it for more than two months, and i’ve only recently switched to this workstation (after an uber hardware failure).

I tried from another machine, and I can copy anything without seeing these errors… The plot thickens…

I’m now convinced that the problem is with my workstation and not the server. Now what?

I noticed a thread by a guy who mentioned that his machine (like mine) was up to date, SP2, but did not have IE7, unlike my other machines. This interested me. Maybe his bug isn’t fixed in the version I’m using? I think it is, it’s quite old…

However, it seems his solution was to upgrade to 3.0.25c and/or change the connection from “netbios-ssn” (port 139) to “microsoft-ds” (port 445)… somehow. I checked on both ends, and appears I’m already using “microsoft-ds” on this machine anyway…

Frustration is starting to kick in…

I’m out of ideas, perhaps I should roll back to an older version, see if that helps? My logs report this as been the previous version:

Jun 07 17:01:55 Updated: samba-common.i386 3.0.10-1.4E.12.2
Jun 07 17:04:29 Updated: samba-client.i386 3.0.10-1.4E.12.2
Jun 07 17:07:44 Updated: samba.i386 3.0.10-1.4E.12.2

I had second thoughts about this when reading the samba change log (rpm -q –changelog samba). There seems to be quite a few security fixes that I’d rather keep…

I’m totally out of ideas… “yum remove samba” it is… However, apparently to install an old version I have to first install a plugin called “yum-allowdowngrade”.

This is getting complicated, especially when it probably isn’t a problem with the server.

I decided to try an updated version instead…

yum remove samba
yum remove samba-common
rpm -i http://charliebrady.org/samba/samba-common-3.0.30-0.i386.rpm

rpm -i ftp://ftp.pbone.net/mirror/www.startcom.org/AS-4.0.0/os/i386/StartCom/RPMS/perl-Convert-ASN1-0.18-3.noarch.rpm
rpm -i http://charliebrady.org/samba/samba-3.0.30-0.i386.rpm
service samba start

It’s installed, and it started, however I appear to have lost my settings.

mv /etc/samba/smb.conf /etc/samba/smb.conf.recent
mv /etc/samba/smb.conf.rpmsave /etc/samba/smb.conf
service smb restart

Testing, and so far so good!

In summary, when I was convinced it was the silly windows based client workstation, instead it was the insanely overly complex linux server. This could all be avoided if redhat/centos wasn’t so keen on backporting and I had the latest version in the first place.

I look forward to the updates.

]]> http://www.hm2k.com/posts/the-specified-network-name-is-no-longer-available-samba/feed 3 Scan to Excel http://www.hm2k.com/posts/scan-to-excel http://www.hm2k.com/posts/scan-to-excel#comments Sat, 24 Oct 2009 12:11:04 +0000 hm2k http://www.hm2k.com/?p=546 This week I’ve mostly been trying to figure out a way to digitalise some of my bank statements to make accountancy and archiving easier.

I figured there must be a way to scan them in using a document scanner, then use OCR software to render the data into Excel.

Of course, nothing does this natively, but there are ways to do it.

There are three parts to this process…

  1. Document scanning
  2. Data format conversion
  3. Optical character recognition (OCR)

For the scanning part I used a Ricoh Aficio MP C4500, which not only acts as a Photocopier, but as a fax machine, network printer and network scanner.

The main part is it’s ability to scan with document processing. This means (once configured) that it would only take a few minutes to scan a few hundred pages, rather than a few hours using a conventional scanner.

This device has the ability to save scanned documents as PDF or TIF. I decided to use the more versatile TIF image format.

I figured it would be easier to convert a standard image format than the proprietary PDF format. I very quickly discovered that it wasn’t that simple.

To begin with I had heard good things about two leading OCR software packages…

  • ABBYY FineReader 9.0
  • Able2Extract Pro 5.0

I found that neither of these software packages would accept TIF files, so I converted each directory of TIF files into a single PDF file using the DreamSys Tiff to PDF Converter, which was a very quick and effective command-line tool for doing the task.

I started trying out Able2Extract as I’d not used it before and I had read good things about it’s PDF to Excel conversion. The problem was it kept throwing up a “Fatal Internal Error #24″ error which I was unable to work around.

I had used ABBYY in the past and it turned out to be very good OCR software for converting to Word at least, however trying to configure it to scan the areas I wanted into Excel seemed almost impossible and took forever.

I decided to go back to the drawing board…

I quickly realised that there were a few “web 2.0″ software as a service tools out there that could assist me…

There’s many universal “online document converters”, a few include…

My issue with most of these is that they would recognise the PDF, but not the images inside the PDF and use OCR to render them. They are only designed to take the text found in a PDF file and extract that to Excel.

I also came across a few other software packages that will also convert from PDF files to Excel…

  • deskUNPDF Professional by docudesk
  • PDF2XL Enterprise by cogniview
  • ExpressConversion Server by adlibsoftware
  • PDF To XLS by verypdf
  • Solid Converter PDF by soliddocuments
  • TotalPDFConverter by coolutils

With all this available proprietary software for converting from PDF to Excel (XLS or CSV) its obvious that it’s a service definitely in demand.

So far I’m waiting on the following:

  • Zamzar and others to email me their conversion of the PDF file I uploaded to them.
  • Able2Extract’s support team to get back to me with regards to the error.
  • For Cogniview PDF2XL Enterprise to download.

Let’s see what gets the job done best, first, if any…

Zamzar is unable to convert from PDF to XLS, directly or indirectly. While finereaderonline will only accept images, not PDF files, which is OK, but may take some time, especially at only 10 pages per day.

Able2Extract were unable to help me with the error unless I sent them the PDF, which I can’t do due to it’s content.

I sent DreamSys an email asking them to send these guys a sample output PDF instead, but I’ve heard nothing back as of yet.

Able2Extract recommended I use their Sonic PDF Creator product to convert the original scanned TIFF files to a PDF which they claim their Able2Extract product will be able to read.

I had a play with Sonic PDF Creator and appeared to be unable to import a directory of TIF files, just each TIF file individually. This seemed like a painstaking process.

I needed to merge the TIF files together into 1 file to make it easier. PTGui Pro is able to stitch TIF files together, however it gave me an error saying:

Error loading TIFF file: Unsupported number of bits per sample (only 8, 16 or 32 are supported) or unsupported sample format. Useless.

Back to square one.

I decided to try my luck with PDF2XL, which after a little teething problem getting a working copy to begin with seemed to be quite a neat package.

PDF2XL was able to detect that the PDF I had given it (the one I created using Tiff 2 PDF) was a scanned document and began performing OCR on the file.

The results were almost prefect, VERY impressive. It seems that PDF2XL is able to do what nothing else could, not even Able2Extract. Don’t waste your time with anything else. It seems that PDF2XL is all you need.

The only thing I will say is I had to tweak the OCR settings a little bit to get it to render the page correctly.

Just when I thought everything was going great, I discovered another issue I had completely overlooked. The output was not in the correct order. I took a look at the original PDF and soon discovered that the images in the PDF were not in the correct order either.

First off, I decided that I should rename all the images based on the sheet number, that way I could be sure that they were correctly ordered. I created a batch file to preview and rename the images, called “imgrename.bat“. I was also able to use this to rename an entire directory at a time.

I tried rebuilding the PDF using DreamSys Tiff to PDF Converter which I had used to create the PDF file in the first place, only to find that it was still out of order and I couldn’t influence the sorting. I had to find something else to do the job…

Enter libtiff and it’s windows counterpart Tiff for Windows by GnuWin32. Once I had a copy of this I was able to write another batch file that would not only combine the TIFF files (with pages) into a single TIFF file (using tiffcp.exe) but I was able to convert that into a PDF using (tiff2pdf.exe). I called this script “tiff2pdf.bat“.

Bingo!

All was successful…

The next task is to firstly, cleanup anything bad in the CSV file left by OCR (cleancsv.bat) then to “normalise” the data…

But that’s another story for another time. We’ve done what we set out to do which was to scan to excel and all is well on that front.

Hope you find this useful. Enjoy!

PDF2XL Enterprise
]]> This week I’ve mostly been trying to figure out a way to digitalise some of my bank statements to make accountancy and archiving easier.

I figured there must be a way to scan them in using a document scanner, then use OCR software to render the data into Excel.

Of course, nothing does this natively, but there are ways to do it.

There are three parts to this process…

  1. Document scanning
  2. Data format conversion
  3. Optical character recognition (OCR)

For the scanning part I used a Ricoh Aficio MP C4500, which not only acts as a Photocopier, but as a fax machine, network printer and network scanner.

The main part is it’s ability to scan with document processing. This means (once configured) that it would only take a few minutes to scan a few hundred pages, rather than a few hours using a conventional scanner.

This device has the ability to save scanned documents as PDF or TIF. I decided to use the more versatile TIF image format.

I figured it would be easier to convert a standard image format than the proprietary PDF format. I very quickly discovered that it wasn’t that simple.

To begin with I had heard good things about two leading OCR software packages…

  • ABBYY FineReader 9.0
  • Able2Extract Pro 5.0

I found that neither of these software packages would accept TIF files, so I converted each directory of TIF files into a single PDF file using the DreamSys Tiff to PDF Converter, which was a very quick and effective command-line tool for doing the task.

I started trying out Able2Extract as I’d not used it before and I had read good things about it’s PDF to Excel conversion. The problem was it kept throwing up a “Fatal Internal Error #24″ error which I was unable to work around.

I had used ABBYY in the past and it turned out to be very good OCR software for converting to Word at least, however trying to configure it to scan the areas I wanted into Excel seemed almost impossible and took forever.

I decided to go back to the drawing board…

I quickly realised that there were a few “web 2.0″ software as a service tools out there that could assist me…

There’s many universal “online document converters”, a few include…

My issue with most of these is that they would recognise the PDF, but not the images inside the PDF and use OCR to render them. They are only designed to take the text found in a PDF file and extract that to Excel.

I also came across a few other software packages that will also convert from PDF files to Excel…

  • deskUNPDF Professional by docudesk
  • PDF2XL Enterprise by cogniview
  • ExpressConversion Server by adlibsoftware
  • PDF To XLS by verypdf
  • Solid Converter PDF by soliddocuments
  • TotalPDFConverter by coolutils

With all this available proprietary software for converting from PDF to Excel (XLS or CSV) its obvious that it’s a service definitely in demand.

So far I’m waiting on the following:

  • Zamzar and others to email me their conversion of the PDF file I uploaded to them.
  • Able2Extract’s support team to get back to me with regards to the error.
  • For Cogniview PDF2XL Enterprise to download.

Let’s see what gets the job done best, first, if any…

Zamzar is unable to convert from PDF to XLS, directly or indirectly. While finereaderonline will only accept images, not PDF files, which is OK, but may take some time, especially at only 10 pages per day.

Able2Extract were unable to help me with the error unless I sent them the PDF, which I can’t do due to it’s content.

I sent DreamSys an email asking them to send these guys a sample output PDF instead, but I’ve heard nothing back as of yet.

Able2Extract recommended I use their Sonic PDF Creator product to convert the original scanned TIFF files to a PDF which they claim their Able2Extract product will be able to read.

I had a play with Sonic PDF Creator and appeared to be unable to import a directory of TIF files, just each TIF file individually. This seemed like a painstaking process.

I needed to merge the TIF files together into 1 file to make it easier. PTGui Pro is able to stitch TIF files together, however it gave me an error saying:

Error loading TIFF file: Unsupported number of bits per sample (only 8, 16 or 32 are supported) or unsupported sample format. Useless.

Back to square one.

I decided to try my luck with PDF2XL, which after a little teething problem getting a working copy to begin with seemed to be quite a neat package.

PDF2XL was able to detect that the PDF I had given it (the one I created using Tiff 2 PDF) was a scanned document and began performing OCR on the file.

The results were almost prefect, VERY impressive. It seems that PDF2XL is able to do what nothing else could, not even Able2Extract. Don’t waste your time with anything else. It seems that PDF2XL is all you need.

The only thing I will say is I had to tweak the OCR settings a little bit to get it to render the page correctly.

Just when I thought everything was going great, I discovered another issue I had completely overlooked. The output was not in the correct order. I took a look at the original PDF and soon discovered that the images in the PDF were not in the correct order either.

First off, I decided that I should rename all the images based on the sheet number, that way I could be sure that they were correctly ordered. I created a batch file to preview and rename the images, called “imgrename.bat“. I was also able to use this to rename an entire directory at a time.

I tried rebuilding the PDF using DreamSys Tiff to PDF Converter which I had used to create the PDF file in the first place, only to find that it was still out of order and I couldn’t influence the sorting. I had to find something else to do the job…

Enter libtiff and it’s windows counterpart Tiff for Windows by GnuWin32. Once I had a copy of this I was able to write another batch file that would not only combine the TIFF files (with pages) into a single TIFF file (using tiffcp.exe) but I was able to convert that into a PDF using (tiff2pdf.exe). I called this script “tiff2pdf.bat“.

Bingo!

All was successful…

The next task is to firstly, cleanup anything bad in the CSV file left by OCR (cleancsv.bat) then to “normalise” the data…

But that’s another story for another time. We’ve done what we set out to do which was to scan to excel and all is well on that front.

Hope you find this useful. Enjoy!

PDF2XL Enterprise
]]> http://www.hm2k.com/posts/scan-to-excel/feed 1 mydigitallife.info stole my code http://www.hm2k.com/posts/mydigitallife-info-stole-my-code http://www.hm2k.com/posts/mydigitallife-info-stole-my-code#comments Wed, 14 Oct 2009 14:02:06 +0000 hm2k http://www.hm2k.com/?p=541 I wrote a little script for an article on fixr.co.uk entitled “Remove OGA Notifier“.

Today I discovered that mydigitallife.info had clearly taken details from my batch file code and used it without giving me or fixr.co.uk any credit at all.

Just compare the actual functions in my removeoga.bat to their RemoveOGA.bat, you can clearly see it’s the same, yet no credit is given at all.

mydigitallife.info is LAME.

- HM2K

Taking someone else’s code and not giving any credit for it is LAME, yes mydigitallife.info is lame and I will be boycotting your site until further notice.

@ECHO off
TITLE mydigitallife.info Boycott
ECHO mydigitallife.info Boycott because they stole my code without giving me credit.
ECHO.
ECHO This will bind mydigitallife.info and www.mydigitallife.info to 0.0.0.0
PAUSE
ECHO 0.0.0.0 www.mydigitallife.info >> %systemdrive%\windows\system32\drivers\etc\hosts
ECHO 0.0.0.0 mydigitallife.info >> %systemdrive%\windows\system32\drivers\etc\hosts

Simply copy this into a notepad window, and save as “mydigitallife_boycott.bat”, then run it.

Anyway, I’ve since improved my batch file to support more versions and languages, so if you copy that too, I’ll be on to you!

Thanks.

Note: If you were to clearly give me and fixr.co.uk credit, I may remove this post.

]]> I wrote a little script for an article on fixr.co.uk entitled “Remove OGA Notifier“.

Today I discovered that mydigitallife.info had clearly taken details from my batch file code and used it without giving me or fixr.co.uk any credit at all.

Just compare the actual functions in my removeoga.bat to their RemoveOGA.bat, you can clearly see it’s the same, yet no credit is given at all.

mydigitallife.info is LAME.

- HM2K

Taking someone else’s code and not giving any credit for it is LAME, yes mydigitallife.info is lame and I will be boycotting your site until further notice.

@ECHO off
TITLE mydigitallife.info Boycott
ECHO mydigitallife.info Boycott because they stole my code without giving me credit.
ECHO.
ECHO This will bind mydigitallife.info and www.mydigitallife.info to 0.0.0.0
PAUSE
ECHO 0.0.0.0 www.mydigitallife.info >> %systemdrive%\windows\system32\drivers\etc\hosts
ECHO 0.0.0.0 mydigitallife.info >> %systemdrive%\windows\system32\drivers\etc\hosts

Simply copy this into a notepad window, and save as “mydigitallife_boycott.bat”, then run it.

Anyway, I’ve since improved my batch file to support more versions and languages, so if you copy that too, I’ll be on to you!

Thanks.

Note: If you were to clearly give me and fixr.co.uk credit, I may remove this post.

]]> http://www.hm2k.com/posts/mydigitallife-info-stole-my-code/feed 2