Author Archives: DevOps

Perl binary search function

Recently I need to implement a binary searching with perl, so I got this code:

 use strict;
 use warnings;
 
 my $want = shift;
 die "$0 number" unless defined $want;
 
 my @list= (3,5,7,11,13,17,19);

 my $pos=bin_search(\@list,$want);
 print "Position: ", defined $pos ? $pos : "undef","\n";

 # binary search
 sub bin_search {
     my $array = shift;
     my $find  = shift;
 
     my ($l,$r)=(0,$#$array);
 
     while ($l<=$r) {
         my $m=int(($l+$r)/2);
 
         if ($find<$array->[$m]) {
             $r=$m-1;
         } elsif ($find>$array->[$m]) {
             $l=$m+1;
         } else {
             return $m;
         }
     }
     return undef;
 } 

It’s smart. Run it this way:

$ perl binarysearch.pl 11
 Position: 3
$ perl binarysearch.pl 13
 Position: 4
$ perl binarysearch.pl 15
 Position: undef 

It’s much faster than loop through the entire array to match the element.

Batch checking the existence of gmail accounts

When you try to register a gmail account, you most probably found all the usernames you desired have been taken.

So, a good tool for batch checking the available mailboxes becomes attractive.

Here I show a method used by myself. It’s a simple perl script:

#!/usr/bin/perl 

 use strict;
 use Gmail::Mailbox::Validate;
 
 my $username = shift || die "$0 username\n";
  
 my $v = Gmail::Mailbox::Validate->new();
 print "$username mailbox exists\n" if $v->validate($username); 

Given the username, this script will tell you if this mailbox at google exists.

For example:

$ ./gmbox wesley9807
 wesley9807 mailbox exists

$ ./gmbox wesley98076
 

The first one tell you username “wesley9807” exists. The second one returns nothing, that username may not be registered. So, you may have the chance to register the username “wesley98076”.

Please notice: The second command returns nothing, it does mean this username has no mailbox at google. But, it still does not mean you can take this username.

For example, google seems keep some good usernames, which have no mailboxes, but you can’t register for them. And, a google user may choose to delete his/her mailbox, but keep the other google service running (google drive etc), so you can not register this mailbox too.

Anyway with this method you can check a lot of usernames quickly. There is no need to try them one by one from google’s registration page.

How to install the required perl module? just use cpanm tool. For example:

$ sudo cpanm Gmail::Mailbox::Validate
 Gmail::Mailbox::Validate is up to date. (0.01) 

The last, you should not abuse it, otherwise google may block your IP or networks.

How to perform a rDNS lookup

rDNS (Reverse DNS) is important for identify an IP address. Some internet service, for instance, sending email from an IP, needs rDNS to be setup correctly.

Here I tell how to perform a rDNS lookup from Linux. There are two simple ways.

The first way, using dig command. The full path is “dig -x IP”, as below:

 $ dig -x 23.95.246.240


 ; <<>> DiG 9.10.3-P4-Ubuntu <<>> -x 23.95.246.240
 ;; global options: +cmd
 ;; Got answer:
 ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11872
 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
 

 ;; OPT PSEUDOSECTION:
 ; EDNS: version: 0, flags:; udp: 512
 ;; QUESTION SECTION:
 ;240.246.95.23.in-addr.arpa. IN PTR
 

 ;; ANSWER SECTION:
 240.246.95.23.in-addr.arpa. 3599 IN PTR 23-95-246-240-host.colocrossing.com.
 

 ;; Query time: 178 msec
 ;; SERVER: 8.8.8.8#53(8.8.8.8)
 ;; WHEN: Tue Feb 23 10:31:28 HKT 2021
 ;; MSG SIZE  rcvd: 104 

In the “ANSWER SECTION”, you will see the PTR record type, the value following that is IP’s rDNS.

The second way, using curl command to query localhost’s rDNS. The full path is “curl -sL hostname.cloudcache.net”, as below:

 $ curl -sL hostname.cloudcache.net
 Your IP: 23.95.246.240, Hostname: 23-95-246-240-host.colocrossing.com. 

As you see, the “hostname:” part is rDNS value for your host’s IP.

Vodafone Germany email migrated to their own platform

I had a vodafone.de email for long days. In the past days, Vodafone Germany hosted their email on Open Xchange, which brought me the worse experience.

Open Xchange is an open source email hosting solution. It has rich features include email, cloud, contact, calendar, push notify, docs etc. It has got a lot of customers during its years of operations. For example, Namecheap, IONOS, Virgilio.it are using their services.

But I don’t like this platform. It is developed by PHP language. It’s slow and not solid. Somethings I can’t open the page, and new email flushing is delayed.

I think many customers at Vodafone.de had the same feelings as me. While I am glad to see recently Vodafone.de has migrated their email service to their own platform, which is the same one as their global platform, such as Vodafone UK.

Their own platform is written by ASP language. It’s clean and fast, and more solid than the OX one. It has the main features of a modern online service, including email, cloud, calendar, contact, note etc.

Comparing to the old OX platform, I much like this new one. I wish vodafone.de gets better and better in their new days.

Send email correctly from Ubuntu VPS

Today getting a VPS becomes more and more cheap. I believe every programmer has his/her own VPS. Sometime you want to run a blog on the VPS, thus you may need to send email such as reports from this VPS too.

To send email from the VPS correctly, you firstly need a clean IP. It requries:

(1) The IP has its own PTR record, for instance:

  # dig -x 64.233.162.27 +short
 li-in-f27.1e100.net. 

Here shows the IP 64.233.162.27 has a valid PTR.

Without PTR you even can’t send email to most big players such as google.

In most cases you can setup IP’s PTR via the provider’s management panel.

(2) The IP has not been listed into any RBL blacklist.

RBL means realtime black lists, they are used for antispam purpose worldwide.

You must make sure your IP isn’t get blacklisted by any RBL.

Please validate it from the site:

http://www.anti-abuse.org/multi-rbl-check/

(3) Your sending domain has the IP added into SPF record.

For example, the sending domain is: myhostnames.com, which wants to send email from the IP 185.213.172.15, thus it should have this SPF record setup in DNS:

myhostnames.com. 300 IN TXT "v=spf1 include:_spf.google.com  ip4:185.213.172.15 ~all" 

Without the SPF setting most recipients may reject your messages, because they don’t know if the domain has the permission to use this IP for sending.

After you have finished setup the IP, you just enable MTA relay on localhost, and send email from your scripts to this relay. MTA relay will route your messages out from localhost to other recipients.

The important step is to make sure you have correct hostname to the VPS, and the hostname should be resolvable, either via DNS or hosts file.

Then, install postfix as relay on localhost:

# apt install postfix
# vi /etc/postfix/main.cf

The last two lines should be changed to:

 inet_interfaces = 127.0.0.1
 inet_protocols = ipv4 

The first line make sure postfix run on localhost only for security reasons. The second line let postfix send mail only via ipv4 protocol, using ipv6 protocol google’s mail servers may reject to talk to us.

Finally restart postfix:

# /etc/init.d/postfix restart

The MTA relay should be working now.

How to send email from within a script? It has many methods. I can show you the perl way below.

We want to install cpanm and the needed modules.

# apt install cpanminus
# cpanm Email::Send::YYClouds
# cpanm LWP::Simple
# cpanm JSON

Then write a script to send statistics report for blog visit.

#!/usr/bin/perl
 use strict;
 use POSIX;
 use Email::Send::YYClouds;
 use LWP::Simple;
 use JSON;
 use utf8;
 
 my $date = strftime "%F",localtime;
 my $subject = "Daily visit statistics - $date";
 
 my $log = "/var/log/apache2/access.log";
 my %hash;
 
 open HD,$log or die $!;
 while (<HD>) {
     # a regex for matching the correct visit IP
     next unless m{^(\d+\.\d+\.\d+\.\d+).*\"GET /.*?/ HTTP/};
     my $ip=$1; 
     $hash{$ip} ++;
 }
 close HD;
 
 my $body = '<table style="width:100%">' . "\n";
 
 for my $ip (sort {$hash{$b} <=> $hash{$a}} keys %hash) {
     my $num = $hash{$ip};
     my $loc ="";
 
     if ($num>=2) {
         # get IP location via public api
         $loc = ip_loc($ip);
     }
 
     $body .= <<DATA;
 <tr>
   <td>$ip</td>
   <td>$num</td>
   <td>$loc</td>
 </tr>
 DATA
 }
 
 $body .= '</table>';
  
 #
 # send email
 #
 my $msg = Email::Send::YYClouds->new();
 $msg->send(recepient => ['[email protected]'],
            sender => '[email protected]',
            smtprelay => 'localhost',
            subject => $subject,
            body => $body,
       );

 #
 # ip query
 #
 sub ip_loc {
     my $ip = shift;
     my $content = get("https://ipapi.co/$ip/json/");
     if (defined $content) {
         my $json = decode_json($content);
         return $json->{city}. ", " . $json->{region}. ", ". $json->{country};
     } else {
         return "";
     }
 }

After you get message, the content looks similar to this table:

66.249.69.4213Ashburn, Virginia, US
66.249.69.4411Ashburn, Virginia, US
66.249.69.469Ashburn, Virginia, US
13.66.139.1304Redmond, Washington, US
113.88.13.164Shenzhen, Guangdong, CN
66.249.73.1003Ashburn, Virginia, US
40.77.167.102Boydton, Virginia, US
217.160.42.1492Frankfurt am Main, Hesse, DE
207.46.13.2312Redmond, Washington, US
185.85.189.2412Bursa, Bursa, TR
66.249.73.1032Ashburn, Virginia, US
121.46.142.2442Guangzhou, Guangdong, CN
62.210.83.42Paris, Île-de-France, FR
13.66.139.1282Redmond, Washington, US
134.159.238.122Kwai Chung, Tsuen Wan District, HK
176.124.231.762Rivne, Rivne, UA
167.114.100.1602Montreal, Quebec, CA
66.249.73.972Ashburn, Virginia, US

Until now, all is done well.