Wednesday, February 18, 2015

Redirecting linux / unix time command output to a file

There may be requirement when you need to redirect unix / linux time command output to a file.

for me, -o option of time command did not work. Hence I used this - run command in a sub-shell and then redirect stdout and stderr into a file


>> All below did not work.

$ time -o /tmp/x date
bash: -o: command not found

real    0m0.001s
user    0m0.001s
sys     0m0.000s

$ time date
Wed Feb 18 08:18:41 UTC 2015

real    0m0.002s
user    0m0.000s
sys     0m0.001s
$ time date |grep real

real    0m0.002s
user    0m0.000s
sys     0m0.001s
$ time date 2>&1 |grep real

real    0m0.003s
user    0m0.000s
sys     0m0.002s
$ (time date )2>&1 |grep real
real    0m0.001s



>> Simple script.

$ cat /var/tmp/dns-check-script.sh
(
echo '----------------------------------------------------'
echo $((date;time -p host mysystem.com 10.10.10.11) &>/dev/stdout )
) >>/var/tmp/dns-check-script.log 2>&1



>> Run script via cron

$ cat /etc/cron.d/dns-check-script.cron
# Run script in every 2 minutes
*/2 * * * * root /var/tmp/dns-check-script.sh



>> This is how log will look like

#  tailf /var/tmp/dns-check-script.log
----------------------------------------------------
Tue Feb 17 12:28:01 GMT 2015 Using domain server: Name: 10.10.10.11 Address: 10.10.10.11#53 Aliases: mysystem.com has address 10.254.10.60 real 0.02 user 0.00 sys 0.00
Tue Feb 17 12:30:01 GMT 2015 Using domain server: Name: 10.10.10.11 Address: 10.10.10.11#53 Aliases: mysystem.com has address 10.254.10.60 real 0.02 user 0.00 sys 0.00



Wednesday, February 11, 2015

How to reduce ext3 or ext4 filesystem in Linux ?

Reducing filesystem is a rare sys admin task and first question asked is - 'why we need to that?' Avoid it if you can. Otherwise, follow below step with standard advise - take backup before attempting it !

Note: You need to unmount filesystem to reduce it !

>> In below example, I have reduced filesystem and LV from 8.9G to 7.9 GB

# df -hP /testfs
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/testvg-testlv  8.9G  151M  8.3G   2% /testfs

# lvreduce -r -L -1G /dev/mapper/testvg-testlv
Do you want to unmount "/testfs"? [Y|n] y
fsck from util-linux-ng 2.17.2
/dev/mapper/testvg-testlv: 11/589824 files (0.0% non-contiguous), 75584/2359296 blocks
resize2fs 1.41.12 (17-May-2010)
Resizing the filesystem on /dev/mapper/testvg-testlv to 2097152 (4k) blocks.
The filesystem on /dev/mapper/testvg-testlv is now 2097152 blocks long.

  Reducing logical volume testlv to 8.00 GiB
  Logical volume testlv successfully resized
  (filesystem will be unmounted before resize and mounted)

# df -hP /testfs
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/testvg-testlv  7.9G  151M  7.4G   2% /testfs


>> I have tested it in following environment.


# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.4 (Santiago)
# uname -r
2.6.32-358.el6.x86_64


Reference : lvreduce

Wednesday, January 14, 2015

Steps to increase SRL volume of Veritas Replicator

Here is brief document to demonstrate how to extend SRL volume of veritas replicator.

Setup

Say you have 2 nodes cluster on Primary site and 2 node cluster on secondary site. VVR configured under Global Cluster (GCO)

Two node SHCFSHA+VVR on Primary site
PrimaryNode01
PrimaryNode02
Two node SHCFSHA+VVR on Secondary site
SecondaryNode01
SecondaryNode02

We have to option:

Option-1: Add a new 5 GB disk to increase SRL vlume
Option-2: Extend existing LUN by 5GB at storage frame and then increase disk size on system



Steps for Option-1: Add a new 5 GB disk to increase SRL volume

Refereed. As it will better IPO scheduling on OS site if now on storage side. Moreover, in this option, we do not deal with existing SRL volume that minimize risk.

Scan disk on all nodes

# for scsi_bus in $(ls /sys/class/scsi_host/); do echo '- - -' > /sys/class/scsi_host/$scsi_bus/scan; done
# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
hparray0_373 auto:cdsdisk    DATA_DISK_1  product_dg    online thinrclm shared
hparray0_995 auto:cdsdisk    -            -            online thinrclm
hparray0_996 auto:cdsdisk    -            -            online thinrclm
hparray0_997 auto:cdsdisk    -            -            online thinrclm
hparray0_998 auto:cdsdisk    SRL_DISK_1  product_dg    online thinrclm shared

# vxdisk -f scandisks

# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
hparray0_373 auto:cdsdisk    DATA_DISK_1  product_dg    online thinrclm shared
hparray0_995 auto:cdsdisk    -            -            online thinrclm
hparray0_996 auto:cdsdisk    -            -            online thinrclm
hparray0_997 auto:cdsdisk    -            -            online thinrclm
hparray0_998 auto:cdsdisk    SRL_DISK_1  product_dg    online thinrclm shared
hparray0_999 auto:none       -            -            online invalid thinrclm

SecondaryNode01# vxdisk list -e
DEVICE       TYPE           DISK        GROUP        STATUS               OS_NATIVE_NAME   ATTR
hparray0_373 auto:cdsdisk   DATA_DISK_1  product_dg   online thinrclm shared sda              tprclm
hparray0_995 auto:cdsdisk   -            -           online thinrclm      sdb              tprclm
hparray0_996 auto:cdsdisk   -            -           online thinrclm      sdc              tprclm
hparray0_997 auto:cdsdisk   -            -           online thinrclm      sdd              tprclm
hparray0_998 auto:cdsdisk   SRL_DISK_1  product_dg   online thinrclm shared sde              tprclm
hparray0_999 auto:none      -            -           online invalid thinrclm sdk              tprclm

Identify disk by udid and confirm disk size. it is to ensure that you got correct disk and playing with correct disk

# vxdisk list hparray0_999|grep udid
udid:      3PARdata%5FVV%5F25B0%5F2AC005C225B0

# fdisk -l /dev/sdk

Disk /dev/sdk: 5.0 GB, 5368709120 bytes
64 heads, 32 sectors/track, 20480 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdk doesn't contain a valid partition table


Initialize disk on MASTER node ( important) of BOTH sites. We do not need to do it on slave node in VxVM/CVM environment.

PrimaryNode01# vxdctl -c mode
mode: enabled: cluster active - MASTER
master: PrimaryNode01

SecondaryNode01# vxdisksetup -i hparray0_999


Confirm disk is online on ALL systems

SecondaryNode01# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
hparray0_373 auto:cdsdisk    DATA_DISK_1  product_dg    online thinrclm shared
hparray0_995 auto:cdsdisk    -            -            online thinrclm
hparray0_996 auto:cdsdisk    -            -            online thinrclm
hparray0_997 auto:cdsdisk    -            -            online thinrclm
hparray0_998 auto:cdsdisk    SRL_DISK_1  product_dg    online thinrclm shared
hparray0_999 auto:cdsdisk    -            -            online thinrclm


Extend DG by adding new disk. Run commad on MASTER node of each site

# vxdg -g product_dg free
DISK         DEVICE       TAG          OFFSET    LENGTH    FLAGS
SRL_DISK_1 hparray0_998 hparray0_998 20480064  322576    -
DATA_DISK_1 hparray0_373 hparray0_373 840957952 2808020688 -

# vxdg -g product_dg adddisk SRL_DISK_2=hparray0_999

# vxdg -g product_dg free
DISK         DEVICE       TAG          OFFSET    LENGTH    FLAGS
SRL_DISK_1 hparray0_998 hparray0_998 20480064  322576    -
SRL_DISK_2 hparray0_999 hparray0_999 0         10441864  -           <<<<< New 5GB disk is part of DG now
DATA_DISK_1 hparray0_373 hparray0_373 840957952 2808020688 -

# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
hparray0_373 auto:cdsdisk    DATA_DISK_1  product_dg    online thinrclm shared
hparray0_995 auto:cdsdisk    -            -            online thinrclm
hparray0_996 auto:cdsdisk    -            -            online thinrclm
hparray0_997 auto:cdsdisk    -            -            online thinrclm
hparray0_998 auto:cdsdisk    SRL_DISK_1  product_dg    online thinrclm shared
hparray0_999 auto:cdsdisk    SRL_DISK_2  product_dg    online thinrclm shared   <<< We see new disk belowng to DG and it is shared

Now resize SRL volume - run command on MASTER node of Primary site. SRL_DISK_2 - is diskname used on both sites with vxdg adddisk command
  -- It will extend SRL volume on secondary site too
  -- It will take couple of minuted pretending as if command has hung !! Keep patience
  -- system load may increase drastically untill command will complete

PrimaryNode01# vradmin -g product_dg resizesrl product_rvg +5G pridiskname=SRL_DISK_2 secdiskname=SRL_DISK_2


Confirm SRL volume has been extended.

# vradmin -g product_dg -l repstatus product_rvg |grep "SRL size"
  SRL size:                   14.65 G

# vxdg -g product_dg free
DISK         DEVICE       TAG          OFFSET    LENGTH    FLAGS
SRL_DISK_1 hparray0_998 hparray0_998 20480064  322576    -
SRL_DISK_2 hparray0_999 hparray0_999 10319872  487968   -           <<<<< New 5GB disk is in use now (offset)
DATA_DISK_1 hparray0_373 hparray0_373 840957952 2808020688 -



Steps for Option-2: Extend existing LUN by 5GB at storage frame and then increase disk size on system


Scan new size of disk and confirm new size on all nodes

# vxdg -g product_dg free
DISK         DEVICE       TAG          OFFSET    LENGTH    FLAGS
SRL_DISK_1 hparray0_998 hparray0_998 20480064  322576    -
DATA_DISK_1 hparray0_373 hparray0_373 840957952 2808020688 -

# for scsi_bus in $(ls /sys/class/scsi_host/); do echo '- - -' > /sys/class/scsi_host/$scsi_bus/scan; done

# vxdisk -f scandisks

# fdisk -l /dev/sdk

Disk /dev/sdk: 15.0 GB, 16106127360 bytes
64 heads, 32 sectors/track, 20480 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdk doesn't contain a valid partition table


Resize disk on each site by running command on MASTER on each site

# vxdctl -c mode
# vxprint -hrt|grep ^dm
# vxdg -g product_dg  free
# vxdisk resize SRL_DISK_1
NOTE : VxVM version < 6.0.5 have bug that can corrupt your data !!
# vxprint -hrt|grep ^dm
# vxdg -g product_dg free
DISK         DEVICE       TAG          OFFSET    LENGTH    FLAGS
SRL_DISK_1 hparray0_998 hparray0_998 20480064  10764440  -   <<<< New size detected
DATA_DISK_1 hparray0_373 hparray0_373 840957952 2808020688 -


Resize SRl volume by running command on MASTER on Primary site 

PrimaryNode01# vradmin -g product_dg resizesrl product_rvg +5G

Confirm SRL volume has been extended.

# vradmin -g product_dg -l repstatus product_rvg |grep "SRL size"
  SRL size:                   14.65 G



References

Disk re-sizing
Resizing the SRL
How to specify specific disks with vradmin when expanding SRL volume 


Friday, November 7, 2014

Can we change disk label from msdos or linux to GPT without losing data?

Can we change disk label ( using fdisk or parted utility) of a diks from msdos/linux to GPT without losing data?

Say we have a 10 GB disk, having disk label msdos(Linux or hex code 83)  and that has ext3 filesystem on 1 GB primary partition.

  • Short answer is NO
  • Long answer is also NO. On my test system, changing label using fdisk preserve data byt filesystem is not longer mountable. Changing label using parted is complete damage - no data and no mounting !!


Here is test output  and explanations


Create a 1 GB partition using fdisk label hex code 83 (Linux)- parted detects it as msdos label

# fdisk -l /dev/sdb

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000a652f

   Device Boot      Start         End      Blocks   Id  System

# fdisk -l /dev/sdb

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-10240, default 1): 
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-10240, default 10240): +1G

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
# fdisk /dev/sdb[3@-l /dev/sdb

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000a652f

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        1025     1049584   83  Linux



Create ext3 filesystem on 1G partition created by fdisk, mount it, copy some dummy data and take note of number of files / total size

# mkfs.ext3 /dev/sdb1
mke2fs 1.41.12 (17-May-2010)
warning: 252 blocks unused.

Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
65664 inodes, 262144 blocks
13119 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=268435456
8 block groups
32768 blocks per group, 32768 fragments per group
8208 inodes per group
Superblock backups stored on blocks: 
32768, 98304, 163840, 229376

Writing inode tables: 0/81/82/83/84/85/86/87/8done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 20 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

# mkdir /dummyfs
# mount /dev/sdb1 /dummyfs
# cp -a /etc/* /dummyfs/
# ls -lR /dummyfs|wc -l
2465
# du -sh /dummyfs
27M /dummyfs




Unmount it and cahnge disklabel to hex code ee (GPT )

# umount /dummyfs

fdisk -l /dev/sdb

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000a652f

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        1025     1049584   83  Linux
# fdisk -l /dev/sdb

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): L

 0  Empty           24  NEC DOS         81  Minix / old Lin bf  Solaris        
 1  FAT12           39  Plan 9          82  Linux swap / So c1  DRDOS/sec (FAT-
 2  XENIX root      3c  PartitionMagic  83  Linux           c4  DRDOS/sec (FAT-
 3  XENIX usr       40  Venix 80286     84  OS/2 hidden C:  c6  DRDOS/sec (FAT-
 4  FAT16 <32m 85="" boot="" c7="" extended="" font="" inux="" nbsp="" prep="" yrinx="">
 5  Extended        42  SFS             86  NTFS volume set da  Non-FS data    
 6  FAT16           4d  QNX4.x          87  NTFS volume set db  CP/M / CTOS / .
 7  HPFS/NTFS       4e  QNX4.x 2nd part 88  Linux plaintext de  Dell Utility   
 8  AIX             4f  QNX4.x 3rd part 8e  Linux LVM       df  BootIt         
 9  AIX bootable    50  OnTrack DM      93  Amoeba          e1  DOS access     
 a  OS/2 Boot Manag 51  OnTrack DM6 Aux 94  Amoeba BBT      e3  DOS R/O        
 b  W95 FAT32       52  CP/M            9f  BSD/OS          e4  SpeedStor      
 c  W95 FAT32 (LBA) 53  OnTrack DM6 Aux a0  IBM Thinkpad hi eb  BeOS fs        
 e  W95 FAT16 (LBA) 54  OnTrackDM6      a5  FreeBSD         ee  GPT            
 f  W95 Ext'd (LBA) 55  EZ-Drive        a6  OpenBSD         ef  EFI (FAT-12/16/
10  OPUS            56  Golden Bow      a7  NeXTSTEP        f0  Linux/PA-RISC b
11  Hidden FAT12    5c  Priam Edisk     a8  Darwin UFS      f1  SpeedStor      
12  Compaq diagnost 61  SpeedStor       a9  NetBSD          f4  SpeedStor      
14  Hidden FAT16 <3 63="" ab="" arwin="" boot="" f2="" font="" hurd="" nbsp="" or="" secondary="" sys="">
16  Hidden FAT16    64  Novell Netware  af  HFS / HFS+      fb  VMware VMFS    
17  Hidden HPFS/NTF 65  Novell Netware  b7  BSDI fs         fc  VMware VMKCORE 
18  AST SmartSleep  70  DiskSecure Mult b8  BSDI swap       fd  Linux raid auto
1b  Hidden W95 FAT3 75  PC/IX           bb  Boot Wizard hid fe  LANstep        
1c  Hidden W95 FAT3 80  Old Minix       be  Solaris boot    ff  BBT            
1e  Hidden W95 FAT1
Hex code (type L to list codes): ee
Changed system type of partition 1 to ee (GPT)

Command (m for help): p

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000a652f

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        1025     1049584   ee  GPT

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
# fdisk -l /dev/sdb

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000a652f

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        1025     1049584   ee  GPT



After changing label from hexcode type 83 ( Linux ) to GPT, mount did not recognoze filesystem type and not able to mount. Did you loose your data?!! Hold one a second !!

# mount /dev/sdb1 /dummyfs/
mount: you must specify the filesystem type
# mount -t ext3 /dev/sdb1 /dummyfs/ 
mount: special device /dev/sdb1 does not exist




Now change disk label back to hexcode 83 (Linux) from GPT ( hex code ee), mount filesystem and your data is there ! Conclusion: Changing disk label using fdisk from msdos/Linux to GPT will preserve data but you will be not able to mount - hence effectively unusable ( I do not know if there is any way to mount it ! )

# fdisk -l /dev/sdb

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 83
Changed system type of partition 1 to 83 (Linux)

Command (m for help): p

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000a652f

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        1025     1049584   83  Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
# fdisk /dev/sdb
#mount -t ext3 /dev/sdb1 /dummyfs
# ls -lR /dummyfs|wc -l
2465

du -sh /dummyfs
27M /dummyfs



Now change disk label from Linux (83) to Linux LVM (8e). You will be able to mount it. Conclusion : Changing label from 83 to 8e preserve data and allow to mount filesystem.

# umount /dummyfs
# fdisk /dev/sdb

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 8e

Command (m for help): p

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000a652f

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        1025     1049584   8e  Linux LVM

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

mount /dev/sdb1 /dummyfs# ls -lR /dummyfs|wc -l
2465

du -sh /dummyfs
27M /dummyfs



Changing label using parted - result loss of partition information in parted print output !! 

# umount /dummyfs/
# parted 
GNU Parted 2.1
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) select /dev/sdb
Using /dev/sdb

(parted) p
Model: VMware Virtual disk (scsi)
Disk /dev/sdb: 10.7GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system  Flags
 1      16.4kB  1075MB  1075MB  primary  ext3         lvm

(parted) mklabel

New disk label type? gpt

Warning: The existing disk label on /dev/sdb will be destroyed and all data on this disk will be lost. Do you want to continue?

Yes/No? Yes

(parted) p
Model: VMware Virtual disk (scsi)
Disk /dev/sdb: 10.7GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start  End  Size  File system  Name  Flags
      ( nothing printed here )

(parted) rescue

Start? 16.4kB

End? 1075MB

searching for file systems... 3% (time left 00:27)
                     searching for file systems... 97% (time left 00:00)

(parted) p
Model: VMware Virtual disk (scsi)
Disk /dev/sdb: 10.7GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start  End  Size  File system  Name  Flags
      ( nothing printed here )

(parted) quit

Information: You may need to update /etc/fstab.



fdisk still recognize partition as GPT, but you will not be able to mount it.

# fdisk -l /dev/sdb

WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.


Disk /dev/sdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        1306    10485759+  ee  GPT
# mount      /dev/sdb1 /dummyfs
mount: you must specify the filesystem type
# mount /dev/sdb1 /dummyfs-t ext3 
mount: special device /dev/sdb1 does not exist



So, once disk label has been changed to GPT using parted, parted do not list partition at all but fdisk still recognize it at GPT.

# parted
GNU Parted 2.1
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.

(parted) / select /dev/sdb
Using /dev/sdb

(parted) p
Model: VMware Virtual disk (scsi)
Disk /dev/sdb: 10.7GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start  End  Size  File system  Name  Flags

(parted) quit



I am sure there will be some way to deal with using parted or cfdisk or some other utility. Do let me know if you know it.



Wednesday, October 29, 2014

Rolling back after minor Redhat RHEL OS upgrade

There is no easy way to rollback if you have upgraded Redhat Linux from say RHEL 6.1 to 6.5.

I have use below steps to roll-back OS to previous version.


1-Collect details of installed rpms before OS upgrade


mkdir /var/tmp/backup
rpm -qa --last > /var/tmp/rpm-qa-last-before.txt 


2-Do following if you want to roll-back rpm to older version

rpm -qa --last > /tmp/rpm-qa-last-after.txt
sdiff -s /var/tmp/rpm-qa-last-before.txt /tmp/rpm-qa-last-after.txt | sed 's/ .* |//g'|awk '{print $1, $2}' >/var/tmp/backup/old-new-rpm-version.txt
echo "yum downgrade `echo $(awk '{print $1}'  /var/tmp/backup/old-new-rpm-version.txt )`"


3-Copy and paste above output to some where for further reference. Execute below to downgrade rpm to older versions

yum downgrade `echo $(awk '{print $1}'  /var/tmp/backup/old-new-rpm-version.txt )`


4-If above fails because of any dependencu error, exclude those package and deal with them later after above steps completed.

 --exclude=package1*,package2*


5- Reboot system


6- Downgrade redhat-release-* rpm to older version if it has not been downgraded

grep redhat-release /var/tmp/rpm-qa-last-before.txt 
rpm -qa |grep redhat-release
yum downgrade redhat-release-xxxx-n.m.-v


This is not perfect process, but works. Preserving OS image of creating some kind of snap/mirror/clone of filesystem is better approach - but this need additional time and resources.


Do you use any better method ?

Thursday, October 9, 2014

Using lftp client to test an ftps connection with ftp servers

FTPS is secure ftp


Few facts


  • This is not FTP over SSL and not FTP over SSH [1] 
  • FTPS use X.509 ( SSL/TLS protocol under FTP)  certificate and validate certificate chain while SFTP use ssh keys.
  • FTP servers exclusively need to be configured for TLS/SSL support. Not all ftp servers support ftps

Does a ftp server support ftps ?


Below command will tell if ftp servers support ftps. You will get warning if ftp server is using self sign certificate or certificate or certificate presented by ftp server cannot be verified by client . In this case last line will be - return code :1 ( though certificate can not be verified, data exchange will still be encypted)

$ openssl s_client -connect dropbox.redhat.com:21 -starttls ftp
CONNECTED(00000003)
---
Certificate chain
 0 s:/2.5.4.15=Private Organization/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/serialNumber=2945436/streetAddress=100 Eas
t Davie St./postalCode=27601/C=US/ST=North Carolina/L=Raleigh/O=Red Hat Inc./CN=dropbox.redhat.com
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended Validation Server CA
 1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended Validation Server CA
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA
---
Server certificate
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
subject=/2.5.4.15=Private Organization/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/serialNumber=2945436/streetAddress=100
East Davie St./postalCode=27601/C=US/ST=North Carolina/L=Raleigh/O=Red Hat Inc./CN=dropbox.redhat.com
issuer=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended Validation Server CA
---
No client certificate CA names sent
---
SSL handshake has read 4571 bytes and written 469 bytes
---
New, TLSv1/SSLv3, Cipher is AES128-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : AES128-SHA
    Session-ID: 5B2D03F4E9682534FA8C83E51FF0D742481B017DC27A90007CD1E3C7DA0FCA9C
    Session-ID-ctx:
    Master-Key: A6E9F2D2F06DDB651C311A6B0C2200CD89DC5EB4D70B75E97C962415F52DF9D67E2069FEC6E21A62680EE7C16706F033
    Key-Arg   : None
    Krb5 Principal: None
    Start Time: 1412838506
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

Test a ftpa connection


Assuming you have an ID ftp_user_id and password is xxxxxx. I am using lftp client . Use lftp-3.7.11-7 or newer version . Older client has Bug

$ cd /tmp
$ cal >test_file_to_upload.txt
$ lftp
debug 9
set ftps:initial-prot ""
set ftp:ssl-allow true
set ftp:ssl-force true
set ssl:verify-certificate no
set ftp:ssl-protect-data true
set ftp:ssl-auth TLS
set ftp:passive-mode true
set ftp:ssl-data-use-keys yes
open dropbox.redhat.com:21
user ftp_user_id  xxxxxx
ls
put test_file_to_upload.txt
ls

---- Resolving host address...
user anonymous  test@something.com
---- 1 address found: 209.132.183.100
lftp dropbox.redhat.com:~> user anonymous  test@something.com
lftp anonymous@dropbox.redhat.com:~> ls
---- Connecting to dropbox.redhat.com (209.132.183.100) port 21

<--- 220="" font="">
---> FEAT
<--- 211-features:="" font="">
<--- font="" nbsp="" ssl="">
<--- font="" nbsp="" tls="">
<--- font="" nbsp="">
<--- font="" nbsp="">
<--- font="" nbsp="">
<--- font="" nbsp="">
<--- font="" nbsp="">
<--- font="" nbsp="">
<--- font="" nbsp="" stream="">
<--- font="" nbsp="">
<--- font="" nbsp="">
<--- font="" nbsp="">
<--- 211="" end="" font="">
---> AUTH TLS
<--- 234="" font="" negotiation.="" proceed="" with="">
---> OPTS UTF8 ON
Certificate:
 Issued by:        C=US,O=DigiCert Inc,OU=www.digicert.com,CN=DigiCert SHA2 Extended Validation Server CA
 Checking against: C=US,O=DigiCert Inc,OU=www.digicert.com,CN=DigiCert SHA2 Extended Validation Server CA
WARNING: Certificate verification: Not trusted
Certificate: C=US,O=DigiCert Inc,OU=www.digicert.com,CN=DigiCert SHA2 Extended Validation Server CA
 Issued by: C=US,O=DigiCert Inc,OU=www.digicert.com,CN=DigiCert High Assurance EV Root CA
WARNING: Certificate verification: Not trusted
<--- 200="" always="" font="" in="" mode.="" utf8="">
---> USER anonymous
<--- 331="" font="" password.="" please="" specify="" the="">
---> PASS test@something.com
<--- 230="" font="" login="" successful.="">
---> PWD
<--- 257="" font="">
---> PBSZ 0
<--- 0.="" 200="" font="" pbsz="" set="" to="">
---> PROT P
<--- 200="" font="" now="" private.="" prot="">
---> PROT P
<--- 200="" font="" now="" private.="" prot="">
---> PASV
<--- 227="" entering="" font="" mode="" passive="">
---- Address returned by PASV seemed to be incorrect and has been fixed
---- Connecting data socket to (209.132.183.100) port 6338
---- Data connection established
---> LIST
<--- 550="" denied.="" font="" permission="">
---- Closing data socket
ls: Access failed: 550 Permission denied. ( it is fine - this website do not allow listing)


Few Issues


Error : gnutls_handshake: Insufficient credentials for that request.
Solution : Use 'set ftp:ssl-data-use-keys yes' and use latest lftp rpm version

Error : gnutls_handshake: A record packet with illegal version was received - while using open ftps://ftp.something.com:21
Solution : Remove ftps:// - just use 'open ftp.something.com:21'


[1] https://www.eldos.com/security/articles/4672.php?page=all
[2] s_cleint documentation 

Wednesday, August 6, 2014

Redhat Satellite - create a software channel with specific minor version

Say you have your organization software channel synced at rhel 6.2 and Redhat parent software channel at latest release 6.5. Now you need to upgrade a rhel 6.1 system to 6.4.

Redhat Satellite provides utility to create a software channel with specific minor release. It does not download all related rpms while creating channel. Required rpms ( simply an older version of your latest rpms !) will already be available in you satellite. It will just create channel and update satellite database table with package IDs.

Below is syntax to create  rhel 6.4 software channel.

# spacewalk-create-channel --help

# spacewalk-create-channel --user=your-satellite-user-name --server=localhost --version=6 --update=U4 --release=Server --arch=x86_64 --destChannel=organisation-rhel6-x86_64-6U4 --name=organisation-rhel6-x86_64-6U4
Password:

You have not specified a source channel, we will try to determine it from inputs
Trying with source channel: rhel-x86_64-server-6
Creating channel, organisation-rhel6-x86_64-6u4, with arch x86_64 3648 packages in source file to push.
Pushing 3648 packages, please wait.
Successfully pushed 3648 packages out of 3648

# spacecmd
Welcome to spacecmd, a command-line interface to Spacewalk.
INFO: Spacewalk Username: your-satellite-user-name
Spacewalk Password:
INFO: Connected to https://localhost/rpc/api as your-satellite-user-name

spacecmd {SSM:0}> softwarechannel_details organisation-rhel6-x86_64-6u4
Label:              organisation-rhel6-x86_64-6u4
Name:               organisation-rhel6-x86_64-6U4
Architecture:       x86_64
Parent:
Systems Subscribed: 0
Number of Packages: 3648

Summary
-------
organisation-rhel6-x86_64-6u4

GPG Key:
GPG Fingerprint:
GPG URL:
spacecmd {SSM:0}>


References
Create a cloned/custom channel from a specific update level on Red Hat Satellite
Create a custom channel to include all errata's and packages released before a specific date