Thursday, January 14, 2016

Dealing with github error: The requested URL returned error: 403 while accessing

Above error means -I have tried to access a git project and I am not authorized .
I will show you one case to get this error and what is fix.

1-Create a github repository


-I have logded in to https://github.com/ using my github using  github credential
-I have created  a repository nagios_plugins (with default options)



2-Create local repository


$ mkdir nagios_plugins
$ cd nagios_plugins
$ echo "# nagios_plugins" >> README.md
$ git init
$ git add README.md
$ git commit -m "first commit"
$ git remote add origin https://github.com/nansari/nagios_plugins.git
$ git push -u origin master


--and I get following error !!

error: The requested URL returned error: 403 while accessing https://github.com/nansari/nagios_plugins.git/info/refs

fatal: HTTP request failed


3-How to fix

--Now, we know, a case where we see this error. Let us fix it. Since url is https - it need some kind of authentication - that we have not configured or provided so for.

--https access to github requires username and password. here is how we can configure git to use a username for each push and pull command. 

--First, I checked what is remote configuration.

$ git remote -v
origin  https://github.com/nansari/nagios_plugins.git (fetch)
origin  https://github.com/nansari/nagios_plugins.git (push)


--Okay, I will modify origin to use my github username.

$ git remote set-url origin https://nansari@github.com/nansari/nagios_plugins.git
$ git remote -v
origin  https://nansari@github.com/nansari/nagios_plugins.git (fetch)
origin  https://nansari@github.com/nansari/nagios_plugins.git (push)
$ cat .git/config


--And, all done ! I am successfully able to push my local branch to remote.

$ git push -u origin master
Password: xxxxxx
Counting objects: 6, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (6/6), 511 bytes, done.
Total 6 (delta 0), reused 0 (delta 0)
To https://nansari@github.com/nansari/nagios_plugins.git
 * [new branch]      master -> master
Branch master set up to track remote branch master from origin by rebasing.


3-A step ahead - pushing local branch to remote


--If you are working in local branch other than master branch, you need to commit change on your local branch and then push your local branch to remote master

$ git push origin local_branch:master
Password: xxxxxx
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 320 bytes, done.
Total 3 (delta 2), reused 0 (delta 0)
To https://nansari@github.com/nansari/
nagios_plugins.git   12b060a..fc3936b  local_branch -> master
 











Saturday, January 9, 2016

SCSI driver Error Handaling (EH) - how multipath behave when few paths of a device fail

Question? 
It is challenge even for most seasoned system admin to explain how the IO will behave when few paths of device fail but some paths are still active. What is your answer ….?
.
.
.
.
.
.
.
.
.
.
.
.  
.
.
.
.
.
.
.
.
.
.
Most people answer this – IO will continue via remaining active paths.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
And…….. that is wrong !!


Let me explain, it with extreme case – say one of remote storage port (rport) of storage frame failed.  So if, there were 2 active paths of device, now device have only one good path and one failed path.

As soon as Linux kernel scsi (sc) driver will detect a failed path, it will quiesce HBA (yes, complete HBA) and wait for all outstanding IO to complete or timeout [1]. Then SCSI layer will activate error handaler. NO IO WILL BE SUBMITTED until error recovery completes – even though one path is failed and still one good paths available. This design is there to avoid any data corruption.

Device Recovery Steps

11-      Abort the command after specified scsi device timeout value defined in /sys/block//device/timeout [1]

When the error handler is triggered, it attempts the following operations in order (until one successfully executes or all options exhausts):

22-      Wait several seconds to hope that remote port become online (if device is Fiber Channel Device – not applicable for SCSI device)
33-      Activate Error Handler and do following in sequence
a.       Reset the device
b.      Reset the bus
c.       Reset the Host

First, scsi driver try to reset device and then bus. If it is not successful, and adapter firmware and device drive decide that adapter has not completed full recovery, adapter will be hard reset. It means, all paths of a disk via that adapter will be unavailable for few moment – irrespective, if they have failed are healthy. Hard reset happens when the I/O is black-holed with NOP response in the fabric.  Since, IO had been frozen by scsi drive, there is no change of IO request drop or data corruption.

Case-1: If all of above will fail, device will be set to offline. It means, complete device is not available via any path. It need manual intervention to look at system/storage logs, find problem, fix it , scan HBA to detect active  paths and make device online/running state.

Case-2: If recovery succeed,  path check heuristics of multipath will mark dead paths as failed. Now, IO to device will continue via remaining active path.


Reduce device recovery time

To reduce overall recovery time, upgrade kernel (to version 2.6.18-371.6.1 or higher) and  device_mapper (to version 0.4.7-63 or higher) to latest release to leverage on time related parameters such as :

11-      scsi driver Error Handaling (EH) timeout – eh_timeout (from default 10 second to 5 seconds)  [4]
22-      HBA port reset time - eh_deadline ( from disable/0 to 5 seconds) [5]
33-      Adpater reset time e.g. Qlogic reset time [2]
Add the following to /etc/modprobe.conf an recreate initrd
options qla2xxx ql2xextended_error_logging=1 qlport_down_retry=10 ql2xloginretrycount=10
44-      Multipath check_timeout  (reduce to 10 seconds from default 60 seconds) [3]

Reference 

[3] /usr/share/doc/device-mapper-multipath-0.4.7/multipath.conf.annotated


Additional Reference

3-      Redhat KB  this and this


Tuesday, January 5, 2016

Kodi - Open Source Home Theatre Software - more I explore, more I get surprised

KODI ( Earlier name XBMC) is Open Source Home Theatre Software


- It runs on Linux, OS X, Windows, iOS, Android - I run OpenElect on Raspberrry Pi
- Have web based, and android / iOS software remote controls - Kore
- 'Play to Kodi' Chrome browser plugin can send you tube video link to Kodi to play it independently
- Third party configuration tools - one such excellent tool is http://www.tvaddons.ag/
- Have tons of plugin - video, audio, configuration, setting, programs etc.
    - http://addons.tvaddons.ag/
    - http://fusion.tvaddons.ag/ [1]
- Some of indian TV /Movie channels provided by 'aj add-ons (new look)'  included in above are liste on below pages.
    - http://www.desirulez.me/forums/41-Tv-Shows
    - http://www.desitvbox.me/
    - http://www.movierulz.to/
- Play entertain media from your Media Center, Desktop, gDrive etc.
- Record live show ( Beaware of legal implication of you storing / download media)
- Support all popular media format of Audio and Video
- UPnP - Universal Plug and Play [2]- Supported by all modern home appliance

[2] https://en.wikipedia.org/wiki/Universal_Plug_and_Play