IP Range for where you are.

Synopsis....

I needed to enable a range of IP's for use in an AWS Security Group so that other developers could access a test website I was working on. Here's how I found the smallest range, rather than opening it the whole world. NOTE: There is a small security issue with this method, it is described below.

All IP's pointing to me have been obfuscated...your eyes aren't going bad. :)

Step 1 - Find your IP

There's lots of ways to do this, but the quick simple way I use is simply going to

https://myip.ms/

And there's your IP!

Step 2 - Find Your IP Range

Now you know your IP. Now click on the WHOIS? icon and it takes you to the next page which shows you everything known about that IP. Who owns it, the IP range that is owned by them, etc.

Here's my general IP range:
X.X.128.0 - X.X.255.255    (32,768 ip)

So I want to figure out the short form of that IP Range.

Scroll down (or search) for CIDR. This is the value you want to use. The whole thing. For example, X.X.128.0/17 (it may be different based on the IP Range from above).



This CIDR is the value I used in the AWS console to denote my range of IP's.

Now the security hole here is that anyone in this set of 32,768 IP's can access my special website (on a specific port) that I setup with this particular rule. I'm ok with this small, if not virtually non-existent hole for my TEST website. Risk vs. Reward. I'll be sure to lock down the production site though.

More information on CIDR and what it means:
https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing

Windows, Nginx, MariaDB...and sometimes SQLServer

I have a couple projects at work that are quite different.

All are using PHP.

Some projects require connecting to MSSql (SQL Server).

Other projects are Drupal based.

I had been using XAMPP for awhile now, but knew I wanted to use Nginx on Windows, so I finally took the leap and got it running.


Here's a great stack to use for Windows, PHP, Nginx, XDebug, and MariaDB:  http://wpn-xm.org/


Now, in my case I needed to interface with a SQL Server database for other projects. The Microsoft Drivers for PHP for SQL Server are located here: https://msdn.microsoft.com/en-us/sqlserver/ff657782.aspx

BUT (at the time of this writing) PHP 32-bit is required for these SQL Server drivers, so make sure you download the correct package from WPN-XM. This was my "gotcha" that took awhile for me to get this running. I was getting "could not find driver" even though the correct versions were available.


I've got some more learning to do, but so far I've got multiple projects running with some accessing MSSQL and others MariaDB, all using Nginx and PHP. The speed difference compared to Apache is remarkably noticeable.

Also for my Drupal projects, I was able to do a simple export from MySQL and then an import into MariaDB. My Drupal sites needed no changes in-order-to run, other than a server config in the nginx.conf file.

Python example to Add task in Todoist

A simple script that will prompt you for a task description and date, then add it to the project in Todoist.


Requires:
* https://github.com/Doist/todoist-python  - View the Readme for installing the todoist-python library
* Python 2.7+
* PIP: https://pypi.python.org/pypi/pip


Code:

import todoist

api = todoist.TodoistAPI('HERE_IS_MY_API_KEY')
task_desc = input('Enter the task: ')
task_time = input('When (optional): ')  
item = api.items.add(task_desc, MY_PROJECT_ID, date_string=task_time)
api.commit()

I just created a simple shortcut in my taskbar and it easily adds tasks to the project ID. I used the Inbox project Id from my account so everything goes in there. I can sort it later.

The "When" parameter recognizes the Todoist date/times such as:

• today
• tomorrow
• 2 weeks
• 1 month

If you press Enter, instead of adding a date, then the task is still added, but with no date.

Multifield - Add value using web service

One little checkbox...that's all it took. Something I didn't understand, but now do just a little better.

Here we are:

1. Using Multifield module
1. Multifield is defined as Unlimited Number of values
2. Using Services with REST server to update a node - adding a value in the list of Multifields
3. Using Postman to PUT to update that node
4. Content type name: test_multi
5. Multifield field name: field_jarod_multi
6. Multifield subfield name: field_jarod_note, Integer

Here's what the Multifield config looks like by default for jarod_multi:

Note the checkbox is on for "Hide blank items". So when editing a node we see this:

That's fine. If I want to add another, I just use the button and we're good.

BUT it doesn't work when adding another through web services. Doing a PUT will give me a return code of 200, but the value is never added.

To do that, uncheck the "Hide blank items" button which gives a view like this when editing a node:

Now in my web services I do a PUT:

The full value for the Key is 

node[field_jarod_multi][und][2][field_jarod_note][und][0][value]

After the PUT, I get the new multifield value added to the node:

Lots of headaches over this one. I just missed the checkbox.  So glad this one is solved.

SMTP Auth Support - Drupal module Gotcha!

In using the SMTP module, I found an error that I really did not expect.

Using the Debug mode of SMTP Auth, I found that the email/password combination I was using was working ok. Auth was always successful. When sending the message, I continued to get "553 Relaying disallowed".

The options for "E-MAIL OPTIONS -> E-mail from address" in the SMTP auth config page were all correct. It should be using this from address to send emails. Nope!

I don't know what made me check the next thing, but I went to "System -> Site Information" and changed the From email address to the one I was using in SMTP auth. Abracadabra, email sending worked.

My Drupal Web Stack - new and improved

It's amazing how sometimes breaking through a major obstacle leads to new and improved technologies.

I had recently realized that one of my servers had been compromised. Hacked into. The best solution I had come up with was to rebuild the server. I quickly did a backup of the databases and all the code and moved them off of the server.

When spinning up a new server, I had the chance to finally move to a different tech stack I had been wanting to use.

1. CentOS 7
2. Nginx
3. MariaDB

This meant no more Apache and no more MySQL. The change between MySQL and MariaDB was really only in the name. There was no change in code or scripts for what I was doing. Awesome!

The Nginx implementation was a lot easier than I thought it was going to be too. I had some Nginx experience from a past contract and knew of the performance increase. Some configuration changes and learning how to setup multiple virtual hosts, and that was that.

At this point, the basic installation seems as fast as my previous Apache + Memcache install.

What is left?

Well this was a very quick implementation in-order-to get my websites moved to a different server. As fast as the stack is right now, I still want to install Memcache as well as Varnish.

Here are a couple of great articles that I used to setup my server and get Drupal 7 running with Nginx and MariaDB.

https://www.rosehosting.com/blog/how-to-install-lemp-linux-nginx-mariadb-php-fpm-on-a-centos-7-vps/

https://www.rosehosting.com/blog/how-to-install-drupal-7-on-centos-7-with-nginx-mariadb-and-php-fpm/

Don't forget to install Drush on your new server. It's too easy:
https://www.drupal.org/node/2132447

A little bit of learning, but I'm very happy with the LEMP and Drupal 7 configuration.

Backup and Migrate - Restore 1 Table

Well I never thought about trying this before, and there was no reason why it wouldn't work, but restoring a single table into a Drupal website using the Backup and Migrate module worked like a charm!

Scenario

I'm using the IP Ranges module on several websites across several servers. Most of the time the websites have content that is specific to certain geographic regions or countries, so getting hit a ton by China, Russia, or Brazil isn't desirable.

Also, adding the IP Ranges entries to each website (or even a new one) is tedious.

Enter Backup and Migrate

On one site, I do all of my manual entries using the IP Ranges interface. Then I can do a backup of the IP Ranges table from this site. The backup will contain info for just IP_RANGES and nothing else. I do not want to break my other sites. I have a Settings Profile for IP Ranges and then I do a manual backup.

On another site, do a restore using that backup file. But wait.....read below.

One Gotcha!

The backup contains a "DROP TABLE IF EXISTS" statement for ip_ranges. Now the script should be fairly quick, but dropping the table while the module is enabled can cause issues for the site if someone visits.

What I like to do is replace the statement in the script with "TRUNCATE" giving: TRUNCATE `ip_ranges`;

double check the rest of the script to make sure you haven't accidentally included other stuff.

Then run the script through the rest of the sites!

I'd rather use Firewall rules, but I need to do more research on that. I've broken access to sites before because I've entered the wrong Firewall rule. :P

Blocking the Spammers

Stop the spammers and bad referrers.

This is a compilation of info I found on various sites.

Add this to the end of your .htaccess file:

SetEnvIfNoCase Via evil-spam-proxy spammer=yes
SetEnvIfNoCase Referer evil-spam-domain.com spammer=yes
SetEnvIfNoCase Referer evil-spam-keyword spammer=yes
SetEnvIfNoCase Via pinappleproxy spammer=yes
SetEnvIfNoCase Referer semalt.com spammer=yes
SetEnvIfNoCase Referer semalt.semalt.com spammer=yes
SetEnvIfNoCase Referer poker spammer=yes

Order allow,deny
Allow from all
Deny from env=spammer

Drupal Modules I use:

ip_ranges - https://www.drupal.org/project/ip_ranges - Block entire range of IP's.

Additional Sites:

http://myip.ms/info/whois/  OR any other popular Whois database. I use this to see the range of IP's I want to exclude. 

This process is a little long for me. Going through the logs and seeing what IP's are hitting bad URL's or protected URL's. Then find the IP range and enter into IP Ranges.  I'd like to see a nice report that uses a Whois database to show where the requests are coming from.

I'm still looking for a better overall solution. For example, I've got to be able to put the rules somewhere in my Apache config to cover all the sites on my server, rather than configuring each one individually, I just haven't taken the time to do it yet. We'll see. :)

Module - Front Page

Do you want different user roles to have different Front pages when they login or go to the Home page?

The Front Page module (https://drupal.org/project/front) works very well for that.  18,109 reported installs at the time of this writing.

If you want token support, 7.x-2.4 and lower will need a patch: https://drupal.org/node/1786128

Front Page handles roles in the reverse order of what you have defined in the Role arrangement and creation page (admin/people/permissions/roles), so Admin role, then Authenticated, and then finally Anonymous role will be processed when implementing a custom front page.

Here's a screenshot of 2 roles, Authenticated and Anonymous.

As an example, for an e-commerce site and with token support, I can redirect a "customer" role to their Orders page.

More advanced users and cases will probably want to use Rules.

Handle potential plural items

Let's say you have a count of items ("widgets") that you want to display.

When you count them, the result could be 0, 1, or more than 1 item.  SO you need to display one of the following:

• 0 widgets
• 1 widget
• 2 widgets

In PHP, you can handle this with the following code:

<?php print $view->total_rows;
 if ($view->total_rows == 1) {
   print " widget"; 
} else {
   print " widgets";
}
 ?> 


Or use Drupals built in function "format_plural" to help out:
<?php print format_plural(max(0, $view->total_rows), '1 widget', '@count widgets'); ?>

JSON for Drupal select list and location

Background

I'm using PhoneGap and DrupalGap to create my first Android app called "2 For Deals" which helps you track and locate deals in stores that are "2 For"  some dollar value.  Location of the item, as well as expiration of the deal are tracked.  Have a gander (with screenshots): https://play.google.com/store/apps/details?id=com.twofordeals

Tech Talk

DrupalGap provides a JavaScript library for your PhoneGap app, as well as a Drupal module which provides services for accessing data through the Drupal site/data.  Great stuff if you want to combine the two.

JSON

Here is the JSON I used to create a new node which includes the following fields:

• field_cost: A Decimal field type using the text field widget
• field_select:  A "List (text)" field type using the "Select list" widget.  Notice the difference between this and some of the other fields.  No index is used.
• A simple Location determined by users' GPS.  This is the location for the node, where 1 and only 1 is allowed.  Not sure if this is correct, but I had to provide at a minimum the locations and location fields.
• Body:  standard body field used in Drupal

{
"node": {
"type": "my-content-type",
"title": "This is my new title ",
"language": "und",
"field_cost": {
"und": {
"0": {
"value": "5"
}
}
},
"field_select": {
"und": {
"value": "key-from-drupal for the select"
}
},
"locations": [{
"locpick": {
"user_latitude": "41.98464",
"user_longitude": "-91.673859"
}
}],
"location": {
"locpick": {
"user_latitude": "41.98464",
"user_longitude": "-91.673859"
}
},
"body": {
"und": {
"0": {
"value": "This is the body of my node"
}
}
}
}
}

For blogging purposes, some of the fields are made more generic. The location is one that exists, but points to the middle of a river.  You won't find me there.  :)

I'm posting this because different pieces caused me different headaches at times.  For example on the mobile app, I wanted to collect as little data as possible about the location, so I needed to find the sweet spot of how to create the location on the server side with as few pieces of data as possible.

TODO:  I will eventually be changing the Location to a node reference.  Which means I will need to find the location(s) closest to the users location, display a list, then submit using that reference.  Fun!!!

I hope this helps someone.  Enjoy!

New Server setup with CentOS 6, Drush and Drupal libs

My compilation of docs and tutorials to help me get a server setup with CentOS 6, Drush and various libs needed for Drupal 7.  Once that is complete and working, a short write-up on getting a Drupal 7 site up and running in less than 2 minutes.

1. Webmin install and setup: http://jarodms-drupal.blogspot.com/2012/07/webmin-on-ec2.html
2. Config Firewall for Webmin:   http://www.webmin.com/firewall.html
3. LAMP Setup: http://library.linode.com/lamp-guides/centos-6
4. Drush in less than a minute: http://danreb.com/content/how-install-drush-centos-linux-drupal-development
5. Additional libraries for Drupal:
1. yum install php-gd
2. yum install php-dom
3. yum install php-mbstring

Additional Steps:

1. Optimize php.ini
2. Depending on server, may need to open firewall for port 80

Install Drupal 7 instance in less than 2 minutes:

1. Create MySQL Database
2. Create MySQL User/pass
3. Grant MySQL privileges for user
4. Config virtual host for new Drupal site and restart Apache

<VirtualHost *:80>

     DocumentRoot "/var/www/html/drupal7"
     ServerName fill-in
     ServerAlias fill-in
     ErrorLog /var/www/logs/drupal7-error.log
     CustomLog /var/www/logs/drupal7-access.log combined
     <Directory "/var/www/html/drupal7">

RewriteEngine on

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !=/favicon.ico
RewriteRule ^(.*)$ index.php?q=$1 [L,QSA]

</Directory>

</VirtualHost>

Use Drush to download Drupal and install site:

1. drush dl 
2. drush site-install standard --account-name=admin --account-pass=admin --db-url=mysql://SiteMySQLUser:SiteMySQLUserPassword@localhost/SiteMySQLDatabase

My Dell Inspiron 1764 - CentOS 6.3 and Broadcom Wireless BCM43224 driver

There is no default driver for Broadcom wireless cards on CentOS 6.3.  Broadcom is apparently getting better at Linux support, so they provide the source you can compile to get the driver.  Here's the awesome and simple writeup that I found.  It really did take just 5-10 minutes of copy/paste/execute.

I won't even try to recreate all the steps in this blog.  Just go to http://elrepo.org/tiki/wl-kmod and follow the instructions.  Worked like a charm!

My setup:
Dell Inspiron 1764 dual-boot into Windows 7 and CentOS 6.3
PCI Device (lspci at terminal) :
Network controller: Broadcom Corporation BCM43224 802.11a/b/g/n (rev 01)

Drupal User Admin module stack

Here's a stack of modules that I like which help a non-administrator manage their own users.

Administer Users By Role:  http://drupal.org/project/administerusersbyrole
Role Delegation:  http://drupal.org/project/role_delegation
Login One Time:  http://drupal.org/project/login_one_time
No Current Password:  http://drupal.org/project/nocurrent_pass

Once installed and enabled, I create a role ("site-owner" for example) and assign it the various permissions.
A few more little tricks and now the Site-Owner can create or cancel users, assign various roles, and send them a One Time Login link if needed.

One caveat I need to investigate...the Administrators (e.g., User 1) are still shown on the list of users (admin/people) and certain roles can be added or removed, but the Administrator role can't be removed and the Admin account can't be edited or canceled.  Really the Site Owner can't do anything harmful to the Admin, but I'd prefer that account wasn't shown at all in the list.  I need a module that will only show the users whose role is less than or equal to the current role.

Drupal 7, Services 3.x , Request and Response

Here is something that I just recently learned regarding request/response from a REST server.
In this case, I am using Drupal 7 with Services 3.x.

It's just one of those things I guess you don't lean until you need to, and I never had yet.  It was difficult to find the info though.

Here is the scenario:

1. You are calling a web service using JSON as the Content Type.
2. You are getting back an XML response.
3. You want to get back a JSON response.

Here's what you need to do.

Goto your services

 

Select "Edit Server"

Make sure the response formatter "json" is selected.  It may already be.

Now in your JSON REQUEST, add the following header:

Accept = "application/json"

For some reason, Drupal was returning the response in XML format by default, I don't know why, but adding this header to the request immediately gave me back a JSON response.

Webmin on EC2

Installing the RPM package of Webmin on EC2 was a 5 step process for me:

*** Verify the latest RPM version at http://www.webmin.com/download.html

1. wget http://prdownloads.sourceforge.net/webadmin/webmin-1.610-1.noarch.rpm
2. sudo rpm -U webmin-1.610-1.noarch.rpm
3. sudo /usr/libexec/webmin/changepass.pl /etc/webmin root foo
1. Change the webmin root password to foo. 
4. sudo /etc/init.d/webmin restart
5. Configure IPTables to allow port 10000: http://www.webmin.com/firewall.html

Webmin will be accessible at http://youserverip:10000

* Works fine on CentOS in Linode and Rackspace.

* Would be nice to install cert for this though.

What's New in Drupal 7

What's New in Drupal 7 | drupal.org

A good page to read...just in case someone asks you.  ;)

'via Blog this'

Getting Drupal updates into TEST

.

1. I create a branch from TEST and call it something like updates. (git checkout -b updates)
1. Now I'm working in the branch updates
2. Create a backup of the database - important and discussed later.
3. Pull the necessary modules or Drupal updates and test it out.
4. Everything look ok?  Then merge the branch to TEST.  
1. git checkout test
2. git merge updates
3. handle any conflicts

Why would you do this rather than pull your Drupal updates right into TEST? Murphy's Law.  If you pull the Drupal updates into TEST first, then somebody is bound to have an issue or item that needs worked on that needs to go into TEST before you had a chance to completely test and verify your Drupal upgrade.  

The approach above helps you get around this.  You're in the middle of testing Drupal updates and someone approaches you with a bug or change that needs to go into TEST first. How do you roll back to the TEST branch? 

1. checkout TEST (git checkout test)
2. Restore the database using the backup you created earlier.
3. You are now at the TEST version you need for the new change, without those Drupal updates that you haven't completely tested.
4. Create your new bug/change branch and continue on the issue that was asked of you.

Drupal - Bootstrap for old PHP script

In a hurry and I have an old PHP application that pulls data from proprietary tables.  I have connected some of the data to a node, but now I need to display the data in a separate window when applicable.

So, using the old and large PHP script, just to get it functioning, I had to bootstrap Drupal.
Place this at the beginning of OldLarge.php


$drupal_dir = getenv("DOCUMENT_ROOT");
$current_directory = getcwd();
chdir($drupal_dir);


require_once './includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

...
Do my Drupal stuff here....db_query() and whatever so that I can get results back from weird tables.
...

Then at the end of  OldLarge.php , add the final code....


chdir($current_directory);

Eventually...some day (which ends up being never) I will convert the OldLarge.php script to something more Drupalized.

CiviCRM "Javascript must be enabled in your browser in order to use the dashboard features"

"Javascript must be enabled in your browser in order to use the dashboard features"

I started getting this error from some of my testers when they were using IE9.  Sure enough, I couldn't access the CiviCRM dashboard or CiviCRM menu (it acted like it was disabled).

Yep, well JavaScript is enabled.

Stats on the site:
CiviCRM version 3.4.8
Latest Drupal (6.24)
Drupal module jquery_update installed (giving Drupal the jQuery version 1.3.2)

Hmmmmmm.....It works FINE with Chrome (16.0.912.77 m), Opera (11.61), Safari (5.1.2), and Firefox (10).

Oh YEAH!  That's right!!!   Internet Explorer doesn't handle a lot of JavaScript files very well.  CSS files are limited to 32(?) I think.  Not sure what the JS limit is.

So on the TEST site I enabled "Optimize JavaScript files" and all works well in IE8/IE9.

Things to look at:
http://drupal.org/project/javascript_aggregator
Great article by Vlad http://shvetsgroup.com/blog/optimizing-javascript-and-css-files-drupal