Ultra light dev setup

Simplification of dev setup:

1. VPS
2. Docker
3. vim/neovim
4. iPad Pro 11 + Keyboard
5. Blink shell

Apps to develop

1. web apps
2. apis
3. text generator using ai
4. number crunching/data science python/R
5. blogs
6. command line games and apps
7. php/js/go/python/ruby/c/c++

How to backup Kubernetes' Config

The easiest solution seems to be following script by Stackoverflow user, Timothy Perez

:

  <pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">#!/bin/bash

# NAMESPACED EXPORTS
 for  ns in  $( kubectl get ns --no-headers | cut -d &#34; &#34;  -f1 ) ;  do
  kubectl --namespace = &#34; ${ ns} &#34;  get -o = json bindings,cm,ep,ev,limits,pvc,po,podtemplates,rc,quota,secrets,sa,svc,controllerrevisions,ds,deploy,rs,sts,localsubjectaccessreviews,hpa,cj,jobs,leases,ev,ds,deploy,ing,netpol,rs,pods,netpol,pdb,roles,rolebindings |  \
       jq '.items[] |
     select(.type!=&#34;kubernetes.io/service-account-token&#34;) |
     del(
         .spec.clusterIP,
         .metadata.uid,
         .metadata.selfLink,
         .metadata.resourceVersion,
         .metadata.creationTimestamp,
         .metadata.generation,
         .status,
         .spec.template.spec.securityContext,
         .spec.template.spec.dnsPolicy,
         .spec.template.spec.terminationGracePeriodSeconds,
         .spec.template.spec.restartPolicy
     )'  >> &#34;./ ${ ns} .json&#34;
 done

# NON-NAMESPACED EXPORTS
kubectl get -o = json cs,ns,no,pv,mutatingwebhookconfigurations,validatingwebhookconfigurations,crds,apiservices,tokenreviews,selfsubjectaccessreviews,selfsubjectrulesreviews,subjectaccessreviews,csr,psp,nodes,psp,clusterrolebindings,clusterroles,pc,sc,volumeattachments |  \
       jq '.items[] |
     select(.type!=&#34;kubernetes.io/service-account-token&#34;) |
     del(
         .spec.clusterIP,
         .metadata.uid,
         .metadata.selfLink,
         .metadata.resourceVersion,
         .metadata.creationTimestamp,
         .metadata.generation,
         .status,
         .spec.template.spec.securityContext,
         .spec.template.spec.dnsPolicy,
         .spec.template.spec.terminationGracePeriodSeconds,
         .spec.template.spec.restartPolicy
     )'  >> &#34;./cluster_non-namespaced_export.json&#34;

        

git error: github.com:abc/xyz.git did not send all necessary objects

I was getting following error when running git pull

git pull
fatal: bad object refs/heads/master 2
error: github.com:abc/xyz.git did not send all necessary objects

I tried running git gc

git gc
error: bad ref for .git/logs/HEAD 2
fatal: bad object refs/heads/master 2
fatal: failed to run repack

The fix was to remove above to files under .git

rm .git/logs/HEAD\ 2
rm .git/refs/heads/master\ 2

        

An Algorithm for Passing Programming Interviews | Hacker News

An Algorithm for Passing Programming Interviews | Hacker News — Read on news.ycombinator.com/item

Kubernetes not passing full path to Nodejs

I spent hours troubleshooting why url path was getting stripped from requests in our Nodejs/Expressjs based app. The reason was this line in Kubernetes’ Ingress:

nginx.ingress.kubernetes.io/rewrite-target : /

Removing it fixed the issue. Check Kubernetes documentation

for more details.

Ask HN: Software Engineer hitting 40: what's next?

Recent thread on Hacker News, https://news.ycombinator.com/item?id=29360119, was very interesting for me especially since I turned 40 this year.

Programming is something I enjoy and pretty good at it. I’m still not at the top of salary range for programmers but this is a tough question. I can get into FAANG but there is still a ceiling of how much one can earn as a software engineer.

This comment

Wealth Lab Pro Earning Play Screener

This is a quick script that I use to find options to buy or sell.

 using  System;
 using  System.Collections.Generic;
 using  System.Text;
 using  System.Drawing;
 using  WealthLab;
 using  WealthLab.Indicators;
 using  Community.Components;

 namespace  WealthLab.Strategies
{
	 public class  MyStrategy  : WealthScript
	{

		 protected override void  Execute()
		{
			DataSeries maFast = EMAModern.Series(Close,  50 );
			DataSeries maSlow = EMAModern.Series(Close,  200 );
			DataSeries maFast_1 = EMAModern.Series(Close,  10 );
			DataSeries maSlow_2 = EMAModern.Series(Close,  50 );
			DataSeries ma = EMAModern.Series(Close,  10 );
			DataSeries maFast_3 = EMAModern.Series(Close,  10 );
			DataSeries maSlow_4 = EMAModern.Series(Close,  50 );

			PlotSeries(PricePane,EMAModern.Series(Close, 50 ),Color.Red,LineStyle.Solid, 2 );
			PlotSeries(PricePane,EMAModern.Series(Close, 200 ),Color.Green,LineStyle.Solid, 2 );
			PlotSeries(PricePane,EMAModern.Series(Close, 10 ),Color.Blue,LineStyle.Solid, 2 );
			PlotSeries(PricePane,EMAModern.Series(Close, 50 ),Color.Red,LineStyle.Solid, 2 );
			PlotSeries(PricePane,EMAModern.Series(Close, 10 ),Color.Blue,LineStyle.Solid, 2 );

			//for(int bar = GetTradingLoopStartBar(201); bar < Bars.Count; bar++)
 			 int  bar = Bars.Count -  1 ;
			{
				 if  (IsLastPositionActive)
				{
					Position p = LastPosition;
					 if  (p.EntrySignal.Contains(&#34;Group1|&#34; ))
					{
						 if  (CrossUnder(bar, maFast_3, maSlow_4))
						{
							SellAtMarket(bar +  1 , p, &#34;Group1&#34; );
						}
					}

				}
				 else
				{
					 if  (maFast[bar] > maSlow[bar])
					{
						 if  (maFast_1[bar] > maSlow_2[bar])
						{
							 if  (Close[bar] < ma[bar])
							{
								 if  (EarningsDate.InWindow( this , bar, &#34;earnings per share&#34; ,  7 ,  0 ))
								{

									BuyAtMarket(bar +  1 , &#34;Group1|&#34; );
								}
							}
						}
					}

				}
			}
		}
	}
}

        

Importing plotly failed. Interactive plots will not work.

Fix:

pip3 install --upgrade plotly

Docker in 5 minutes

I have been using Docker and Kubernetes for several years now but never really took any time to really read about it. There were a lot of gaps in my understanding of how Docker works and what really is possible with it. Just recently my employer started to offer Udemy’s subscription, so I took Hands on Docker course. Here are some of the important things I learned in this course.

Json_error_ctrl_char

I was using mcrypt to encrypt json_encoded data.

On other side, mycrypt was able to decrpyt data but json_decode was not working. It would throw JSON\_ERROR\_CTRL_CHAR error.

However, without mcrypt, json_decode worked flawlessly.

After some trial and error, I found that applying trim function after decrypting data would let me use json_decode without any issues.

Find Large Directories on AIX

I usually run it from under a partition which is running out of space. It can take a while.

du -g | sort -n -r

Error Xcode Select Error Tool Xcodebuild Requires Xcode Active Developer Directory Command Line Tools Instance

Playing with Cordova, I was getting this error when building iOS version:

Error: xcode-select: error: tool 'xcodebuild' requires Xcode,
but active developer directory is a command line tools instance

Even full install of Xcode didn’t fix this error.

The solution was to run following command to use full Xcode instead of command line tools version that I had installed earlier:

sudo xcode-select --switch /Applications/Xcode-beta.app/Contents/Developer

Setup Outgoing Email on Lightsail Ubuntu VPS

I followed instructions here: https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-postfix-as-a-send-only-smtp-server-on-ubuntu-18-04

Everything seemed okay but email were not getting delivered. Logs showed me that smtp connections were timing out:

tail -f /var/log/mail.log
May  3 12:28:10 postfix/smtp[3160]: connect to gmail-smtp-in.l.google.com[172.217.197.27]:25: Connection timed out
May  3 12:28:10  postfix/smtp[3160]: connect to alt1.gmail-smtp-in.l.google.com[2800:3f0:4003:c00::1a]:25: Network is unreachable
May  3 12:28:40 postfix/smtp[3160]: 4984C41A1E: to=<xxxx@gmail.com>, relay=none, delay=3246, delays=3186/0.01/60/0, dsn=4.4.1, status=deferred (connect to alt2.gmail-smtp-in.l.google.com[2a00:1450:400b:c00::1a]:25: Network is unreachable)

        

Files Empty in html form?

Put enctype="multipart/form-data" in your form tag.

hsts Neterr_cert_common_name_invalid

Without full understanding, I had enabled HSTS on amerkhalid.com with option includeSubDomains. I had a subdomain that was used as “Custom Domain” to SmugMug site. After enabling HSTS, these subdomains started to throw NET::ERR_CERT_COMMON_NAME_INVALID.

The fix is of course simple, don’t use includeSubDomains. But that opens up your top level domain to man in middle attacks.

For now, I decided to follow the best practices and leave includeSubDomains enabled. And decided to not use custom domain for my SmugMug site.

TypeError: require.extensions.hasOwnProperty is not a function

While playing with https://github.com/alexa/interactive-adventure-game-tool, I ran into following error:

> interactive-adventure-game-tool@1.0.0 start /Users/amer/alexa/interactive-adventure-game-tool
> node node_modules/gulp/bin/gulp.js

/Users/amer/alexa/interactive-adventure-game-tool/node_modules/require-dir/index.js:97
            if (!require.extensions.hasOwnProperty(ext)) {
                                    ^

TypeError: require.extensions.hasOwnProperty is not a function
    at requireDir (/Users/amer/alexa/interactive-adventure-game-tool/node_modules/require-dir/index.js:97:37)
    at Object.<anonymous> (/Users/amer/alexa/interactive-adventure-game-tool/gulpfile.js:1:85)
    at Module._compile (module.js:660:30)
    at Object.Module._extensions..js (module.js:671:10)
    at Module.load (module.js:573:32)
    at tryModuleLoad (module.js:513:12)
    at Function.Module._load (module.js:505:3)
    at Module.require (module.js:604:17)
    at require (internal/module.js:11:18)
    at Liftoff.handleArguments (/Users/amer/alexa/interactive-adventure-game-tool/node_modules/gulp/bin/gulp.js:116:3)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! interactive-adventure-game-tool@1.0.0 start: `node node_modules/gulp/bin/gulp.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the interactive-adventure-game-tool@1.0.0 start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/amer/.npm/_logs/2018-01-02T05_12_24_832Z-debug.log

        

Disable Ping in Linux

echo 1 >/proc/sys/net/ipv4/icmp_echo_ignore_all

Troubleshooting Kubernetes Ingress

Setting up Ingress is an easy process but when it doesn’t work it gets really painful. First, make sure you have Ingress Controller setup correctly. This is in addition to Ingress resource and should be automatically setup by your cloud provider. When I was trying to setup a Kubernetes cluster on IBM Cloud, I ran into a lot of issues. It seems due to my permissions level something went wrong during provisioning of Kubernetes and Ingress Controller was not setup correctly. I went through a lot of steps including recreating alb , ingress resources, etc. The final fix was to login to IBM Cloud as superuser to change Access Policies for the cluster. Once did that, everything just worked magically.

iPad Pro as main machine after 2 years

I got my iPad Pro at the end of 2017. Before buying it, I had read many blog posts by various developers who were using iPad for programming and web development. But soon after I bought it, I realized that serious web development on iPad is hard. The biggest issue was debugging JavaScript and CSS without web tools. At first, I was having buyer’s remorse but I loved the form factor of iPad and loved drawing on it. I am used to reading on Kindle and iPad was just too heavy for long reading sessions but it was very decent reading device for quick reading sessions. So I kept it.

Git Simplified

One of most common technology that new professional developers struggle with is git.

Many junior developers are eager to start using all of the powerful commands of git and usually end up getting more confused.

Here are a few commands that I recommend to anyone learning git for the first time:

clone

This downloads entire git repo from a remote server, usually. You will get all revision history and branches.

Tools of Trade

I use 13-inch MacBookPro for personal projects and 15-inch for work. My personal MBP would be last traditional laptop, hopefully. Since most of my work can easily be done on Linux VPS. I have been using 10.5-inch iPad Pro with Blink shell for most my side projects. I like the form factor of iPad Pro and I am getting better at Linux administration. Now I am also setting up Docker images so I can bring up new images as needed.

You don't need a web developer

Occasionally, when people find out I am a programmer, they ask me if I will build  them a website or an app on the side. As I spend all day programming at my work, I rarely have motivation to continue programming after work.

However, talking to most people, I realize they don’t need a programmer; most of their needs can be met by a simple SaaS solution. Recently, I directed a few friends to WordPress.com, SmugMug, Shopify, etc. They all were happy with the results and ease of use. Also I am glad they talked to me because one guy was ready to spend a few thousands on a developer for a WordPress-based site.

WordPress vs Hugo

Managing WordPress can get time consuming. I have tried to move to static website several times but kept going back to WordPress. But there are several advantages of static sites generators that I finally moved for good. I am also advising a lot of my clients to use Hugo especially when they know that they will rarely ever update their sites.

Here are some of main advantages of Hugo (or other static site generators) vs WordPress and other CMS.

iPad Pro for Programming & Fun

I finally bought 10.5" iPad Pro (Affiliate Link). It was mostly an impulse purchase. When first I started to use iPad, it felt blah. It is hard to find good apps or what apps you might want to try out.

My main goals with iPad was to have a really small laptop replacement. So I was hoping for a decent code editor. There are some code editing apps but there is no way to try them out before purchasing. So I am spending a lot of time reading reviews before I purchase any app.

ReflectionException: Class Tests\Unit\Symfony\Component\HttpKernel\Exception\NotFoundHttpException does not exist

Running unit tests in Laravel, I was getting this error:

ReflectionException: Class Tests\Unit\Symfony\Component\HttpKernel\Exception\NotFoundHttpException does not exist

The issue was missing root backslash, make sure you have expected exception like:

$this->expectException(\Symfony\Component\HttpKernel\Exception\NotFoundHttpException::class);

Upgrading to PHP 7.0 on Ubuntu 14

I upgraded PHP to version 7.0 on Ubuntu box. Running php -v on shell would show it as version 7.0. But Apache was still using PHP 5.6. I tried various methods to update settings for Apache but nothing worked until I issued following commands:

sudo a2dismod php5.6
sudo a2enmod php7.0
sudo service apache2 restart

Source: PHP 7.0 (and 5.6) on Ubuntu | LornaJane

Setting Up Laravel on MacOS

After 3 years, I am using Laravel again at work. Laravel has extensive documentation but sometimes it can be a bit verbose. Here are command to get you started as soon as possible. This assumes pretty much fresh install of MacOS.

Install Homebrew

Check the official site for latest command

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Install PHP 7.1

brew install homebrew/php/php7

Install MariaDB

brew install mariadb

Magento – Get All Products with Categories in a Flat View

     SELECT
        w1.website_id,
        w1.name  as  website_name,
        s1.store_id,
        s1.name  as  store_name,
        p1.entity_id  as  product_id,
        p1.sku,
        pname.value  as  product_name,
        url.value  as  url_path,
        small_image.value  as  small_image,
        msrp.value  as  msrp_price,
        price.value  as  price,
        p1.created_at  as  product_created_at,
        p1.updated_at  as  product_updated_at,
        visibility.value  as  visibility,
        pstatus.value  as  status,
         case
             when
                (pstatus.value  =   1
                     and  visibility.value  >   1 )
             then
                 1
             else   0
         end as  enable_flag,
        c1.entity_id  as  category_id,
        cname.value  as  category_name,
        c1.parent_id,
        c1.created_at  as  category_created_at,
        c1.updated_at  as  category_updated_at
     FROM
        catalog_product_entity p1
             inner join
        eav_attribute p_attr  ON  p1.entity_type_id  =  p_attr.entity_type_id
             and  p_attr.attribute_code  =  'name'
             inner join
        catalog_product_entity_varchar pname  ON  pname.entity_id  =  p1.entity_id
             and  pname.attribute_id  =  p_attr.attribute_id
             inner join
        eav_attribute p_attr2  ON  p1.entity_type_id  =  p_attr2.entity_type_id
             and  p_attr2.attribute_code  =  'url_path'
             inner join
        catalog_product_entity_varchar url  ON  url.entity_id  =  p1.entity_id
             and  url.attribute_id  =  p_attr2.attribute_id
             and  pname.store_id  =  url.store_id
             inner join
        eav_attribute p_attr3  ON  p1.entity_type_id  =  p_attr3.entity_type_id
             and  p_attr3.attribute_code  =  'small_image'
             inner join
        catalog_product_entity_varchar small_image  ON  small_image.entity_id  =  p1.entity_id
             and  small_image.attribute_id  =  p_attr3.attribute_id
             and  pname.store_id  =  small_image.store_id
             inner join
        eav_attribute p_attr4  ON  p1.entity_type_id  =  p_attr4.entity_type_id
             and  p_attr4.attribute_code  =  'msrp'
             inner join
        catalog_product_entity_decimal msrp  ON  msrp.entity_id  =  p1.entity_id
             and  msrp.attribute_id  =  p_attr4.attribute_id
             and  pname.store_id  =  msrp.store_id
             inner join
        eav_attribute p_attr5  ON  p1.entity_type_id  =  p_attr5.entity_type_id
             and  p_attr5.attribute_code  =  'price'
             inner join
        catalog_product_entity_decimal price  ON  price.entity_id  =  p1.entity_id
             and  price.attribute_id  =  p_attr5.attribute_id
             and  pname.store_id  =  price.store_id
             inner join
        eav_attribute p_attr6  ON  p1.entity_type_id  =  p_attr6.entity_type_id
             and  p_attr6.attribute_code  =  'visibility'
             inner join
        catalog_product_entity_int visibility  ON  visibility.entity_id  =  p1.entity_id
             and  visibility.attribute_id  =  p_attr6.attribute_id
             and  pname.store_id  =  visibility.store_id
             inner join
        eav_attribute p_attr7  ON  p1.entity_type_id  =  p_attr7.entity_type_id
             and  p_attr7.attribute_code  =  'status'
             inner join
        catalog_product_entity_int pstatus  ON  pstatus.entity_id  =  p1.entity_id
             and  pstatus.attribute_id  =  p_attr7.attribute_id
             and  pname.store_id  =  pstatus.store_id
             inner join
        catalog_category_product ccp  ON  ccp.product_id  =  p1.entity_id
             inner join
        catalog_category_entity c1  ON  c1.entity_id  =  ccp.category_id
             inner join
        eav_attribute c_attr  ON  c1.entity_type_id  =  c_attr.entity_type_id
             and  c_attr.attribute_code  =  'name'
             inner join
        catalog_category_entity_varchar cname  ON  cname.entity_id  =  c1.entity_id
             and  cname.attribute_id  =  c_attr.attribute_id
             and  pname.store_id  =  cname.store_id
             inner join
        catalog_category_product_index store1  ON  store1.product_id  =  p1.entity_id
             and  store1.category_id  =  c1.entity_id
             inner join
        core_store s1  ON  store1.store_id  =  s1.store_id
             inner join
        core_website w1  ON  s1.website_id  =  w1.website_id

        

OpenShift Error: Layer 7 Wrong Status, Invalid Response

I have been playing with OpenShift for past several hours. It looks great. But a while back I started to get 503 Internal Server error.

When I checked logs using rhc tail , I saw this error: Layer 7 Wrong Status, Invalid Response 404.

I spent an hour or so troubleshooting. Turns out the issue was Netbeans had added src/main/web/app/WEB-INF/jboss-web.xml when I ran the application locally on my machine. I committed this file, thinking I might need it. Deleting it from repo fixed the issue.

How to Tell if a Number Is Whole in Php

if ($num == (int) $num) {
    // It's whole
} else {
    // It's not
}

Filter Some Keys in Multi Dimensional Arrays in Php

/**
 * Cleans up multi-dimensional arrays.
 * 1st dimension is a simple index
 * 2nd dimension includes the desired keys
 *
 * @param mixed $array
 * @param mixed $keysToInclude
 */
public function cleanUpArray($array, $keysToInclude) {
    $returnArray = array();
    $i = 0;

    foreach($array as $item){
        foreach($keysToInclude as $key){
            $returnArray[$i][$key] = $item[$key];
        }

        $i++;
    }

    return $returnArray;
}

        

MySQL Update If Exist Else Insert Procedure

Are you tired of checking data in your code before inserting? Well MySQL procedures are here to rescue. This simple procedure shows how you can do that:

CREATE PROCEDURE   ` update_insert_user ` (  IN  uid2 int  )
 BEGIN
 DECLARE  last_login2 DATETIME;
 SELECT   ` last_login ` INTO  last_login2  FROM   `  user  ` WHERE   ` uid `   =  uid2  LIMIT   1 ;
 IF  last_login2  IS NULL THEN
 INSERT INTO   `  user  `  ( ` uid ` , ` last_login ` )  values  (uid2, now());
 ELSE
 UPDATE   `  user  ` SET   ` last_login `   =  now()  WHERE   ` uid `   =  uid2  LIMIT   1 ;
 END IF ;
 END

        

$this->db->save_queries = FALSE;