MERGING PDF: UBUNTU 20.04 LTS

This small post shares a one line command in Ubuntu terminal to merge to PDFs. While for the regulars,merging the same online would seem to be the quickest and the best but there are times when you wish your PDFs are not compromised by third party websites offering these mergers for free...so why not use a one line command with the help of a tool pdftk.

The command to install the same is 

sudo apt-get install pdftk

and the command thereafter to merger the PDFs like three files 1.pdf, 2.pdf and 3.pdf is as seen below:

 pdftk 1.pdf 2.pdf 3.pdf cat output 4.pdf

where 4.pdf is the merged final output file.

That's all...no third party website reliance for private files.

REMOVE METADATA FROM IMAGES: UBUNTU with exiftool utility

Just few commands in Ubuntu to check the meta data and thereafter remove the same.I am using a utility exiftool which does not come inbuilt to most of the Linux systems,but we need to install them.

INSTALL COMMAND

sudo apt-get install libimage-exiftool-perl

USAGE COMMAND

exiftool -all= IMG_20200504_215424.jpg

(The above command creates a separate copy of the original file and the new file which is created with removed metadata)

or

exiftool -overwrite_original -all= IMG_20200504_215424.jpg 

(The above command overwrites the original file with the new file created with removed metadata)

Showing you output of one file prior to removal of meta data and thereafter removing the meta data

ORIGINAL FILE

kabali@Kabali:~/Desktop/ddd$ jhead IMG_20200504_215424.jpg
File name    : IMG_20200504_215424.jpg
File size    : 3588382 bytes
File date    : 2021:01:31 14:10:13
Camera make  : Xiaomi
Camera model : Mi A2
Date/Time    : 2020:05:04 21:54:24
Resolution   : 4000 x 3000
Orientation  : rotate 90
Flash used   : No
Focal length :  4.1mm  (35mm equivalent: 4mm)
Exposure time: 0.050 s  (1/20)
Aperture     : f/1.8
ISO equiv.   : 1000
Whitebalance : Auto
Metering Mode: center weight
GPS Latitude : ? ?
GPS Longitude: ? ?
JPEG Quality : 98

REMOVED METADATA FILE

kabali@Kabali:~/Desktop/ddd$ jhead IMG_20200504_215424.jpg

File name    : IMG_20200504_215424.jpg
File size    : 3556853 bytes
File date    : 2021:01:31 14:30:08
Resolution   : 4000 x 3000
JPEG Quality : 98

 

REMOVING META DATA FROM VIDEO FILES : UBUNTU TERMINAL COMMANDS

Hello everyone, this post focuses on removing metadata from a video file using ffmpeg inbuilt utility in Ubuntu OS. The following command includes the original file MAH08207.mp4 being stripped of metadata and being output into a new file as output.mp4.

ffmpeg -i MAH08207.mp4 -map 0 -map_metadata -1 -c copy output.mp4

The output of the command is as below:

ffmpeg version 4.2.4-1ubuntu0.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
  configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'MAH08207.mp4':
  Metadata:
    major_brand     : MSNV
    minor_version   : 22675568
    compatible_brands: MSNVmp42isom
    creation_time   : 2020-11-09T00:48:18.000000Z
  Duration: 00:00:44.16, start: 0.000000, bitrate: 12146 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709/bt709/iec61966-2-4), 1440x1080 [SAR 4:3 DAR 16:9], 12016 kb/s, 25 fps, 25 tbr, 25k tbn, 50 tbc (default)
    Metadata:
      creation_time   : 2020-11-09T00:48:18.000000Z
      handler_name    : Video Media Handler
      encoder         : AVC Coding
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      creation_time   : 2020-11-09T00:48:18.000000Z
      handler_name    : Sound Media Handler
Output #0, mp4, to 'output.mp4':
  Metadata:
    encoder         : Lavf58.29.100
    Stream #0:0: Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709/bt709/iec61966-2-4), 1440x1080 [SAR 4:3 DAR 16:9], q=2-31, 12016 kb/s, 25 fps, 25 tbr, 25k tbn, 25k tbc (default)
    Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 127 kb/s (default)
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame= 1104 fps=0.0 q=-1.0 Lsize=   65489kB time=00:00:44.13 bitrate=12154.5kbits/s speed= 540x    
video:64774kB audio:688kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.039428%

The size of the original file and the stripped of file is

-rw-r--r-- 1 kabali kabali 67050400 Nov  9 06:19 MAH08207.mp4
-rw-rw-r-- 1 kabali kabali 67060450 Jan 29 20:23 output.mp4 

Thus difference between the files is the meta data removed.

I am also showing the output of ffprobe utility run on the two files to show you the difference of meta data removed too.

ffprobe MAH08207.mp4

 ffprobe version 4.2.4-1ubuntu0.1 Copyright (c) 2007-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
  configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'MAH08207.mp4':
  Metadata:
    major_brand     : MSNV
    minor_version   : 22675568
    compatible_brands: MSNVmp42isom
    creation_time   : 2020-11-09T00:48:18.000000Z
  Duration: 00:00:44.16, start: 0.000000, bitrate: 12146 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709/bt709/iec61966-2-4), 1440x1080 [SAR 4:3 DAR 16:9], 12016 kb/s, 25 fps, 25 tbr, 25k tbn, 50 tbc (default)
    Metadata:
      creation_time   : 2020-11-09T00:48:18.000000Z
      handler_name    : Video Media Handler
      encoder         : AVC Coding
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      creation_time   : 2020-11-09T00:48:18.000000Z
      handler_name    : Sound Media Handler

ffprobe output.mp4 

 ffprobe version 4.2.4-1ubuntu0.1 Copyright (c) 2007-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
  configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'output.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.29.100
  Duration: 00:00:44.16, start: 0.000000, bitrate: 12148 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709/bt709/iec61966-2-4), 1440x1080 [SAR 4:3 DAR 16:9], 12016 kb/s, 25 fps, 25 tbr, 25k tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      handler_name    : SoundHandler

SPLIT VIDEOS IN UBUNTU: SINGLE LINE COMMAND{ffmpeg utility}

Sharing here just a single line command of ffmpeg utility which can split a video with time duration's specified.So if your video is suppose 05:03(min:sec) long and you wanna split it into two sizes with part 1 as 00:00 to 02:28 and the other part 2 as 02:28 to 05:03,the commands to be executed will be

ffmpeg -i VID_20210129_115611.mp4 -ss 00:00:00 -t 00:02:28 part1.mp4

ffmpeg -i VID_20210129_115611.mp4 -ss 00:02:28 -t 00:05:03 part2.mp4

where VID_20210129_115611.mp4 is the main file and part1.mp4/part2.mp4 are the respective output files.

MERGE VIDEOs IN UBUNTU: TWO TERMINAL COMMANDS

Hi guys...no time waste here...I m mentioning two commands to simply merge your multiple videos quickly in an Ubuntu OS.No GUI or other applications to download or use. The ffmpeg command inbuilt to Ubuntu easily merges without hassles.

So,the first command is 

find "$(pwd)" -type f -name '*.mp4' -printf "file '%p'\n" | tee mp4_compile.txt

The above command will simply find your all mp4 files in the directory wherein you run this command in the terminal directory and compile them into one text file. This text file is named here as mp4_compile.txt. You can name it any thing.

The second command is as given below:

ffmpeg -f concat -safe 0 -i mp4_compile.txt -c copy merged.mp4

This command will merge all the files compiled in the txt file and merge into one output file names as merged.mp4

Thats all...I actually went through a lot of softwares and applications...tried online merge too but all had some kind of limitation in size...till I found these commands.So happy sharing and happy merging.

NOTHING BLOCKCHAIN ABOUT IT :-)

SPLIT LARGE FILES INTO SMALLER PARTS & MERGE : FOR EASY MIGRATION in LINUX SYSTEMS

1.    Routine troubleshooting in system handling gives us a way to explore and try new options.Recently tried a new way to migrate large files(around ~17 GB in my case here ). Although there are easier options available by simply copying in a external device,but in my case external devices were not allowed so attempted to split the large file into 3 GB size files and then finally could merge them....and could also verify the hash....and it was absolutely right...sharing few screen shots :

2.    The entire thing of splitting and merging gets over in simple two commands of SPLIT & CAT

 > split -b file.name
 > cat x* > file.name

3.   I have a .vdi virtual image file which is approx 16 GB in size.The file details are seen in the screen shot below :

Here ..before I go for the split...I have taken the hash of the same to compare it later after i merge the split parts...to check the originality of files and I get the hash of the file as bb867749cf4c0325abe145a0998e3b04

 

In the screen shot below I use the split command,which is inbuilt to UBUNTU and typically all Linux systems....I split the file in 3GB parts...

 

This screen shot shows the watch screen shot of the populating split parts...

 This screen shot shows the watch screen shot of the populating split parts...

 and here i get the final .vdi file as well m able to match the hash as same ie bb867749cf4c0325abe145a0998e3b04

How to install Graylog on Ubuntu 18.04.1 LTS ?

1.    Graylog2 is a powerful log management and analysis tool that has many use cases, from monitoring SSH logins and unusual activity to debugging applications. It is based on Elasticsearch, Java, MongoDB, and Scala.The installation is not exactly to say simple since a lot of integration goes behind involving different applications including Elasticsearch, Java, MongoDB, and Scala. But in few steps and little modifications in default configuration files,the same can be run with ease.

2.   Below are terminal commands with screen shots to achieve the same running

Firstly,the system that I m installing the GRAYLOG on ..the specs are as below

 Then coming to the root terminal...from where all the commands will be executed for installation.

$ apt update && apt upgrade

 Next we install the JDK module

$ apt install apt-transport-https uuid-runtime pwgen openjdk-8-jre-headless

 This is followed by installing ELASTICSEARCH

$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -

 $ echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list

Next we install
 $ apt update && apt install elasticsearch

This is followed by enabling the service and starting

$   systemctl enable elasticsearch $   systemctl start elasticsearch

Amend cluster.name as graylog

 Followed by these lines at the EOF

script.inline: false

script.indexed: false

script.file: false

 

This is followed by

$ systemctl restart elasticsearch

 

Now install the key

$ apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5 

 Then MANGODB repository

$ echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.6 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list

 Install mongodb

$ apt update && apt install mongodb-org

$   systemctl start mongod

$    systemctl enable mongod

$ wget https://packages.graylog2.org/repo/packages/graylog-2.5-repository_latest.deb
$ dpkg -i graylog-2.5-repository_latest.deb
$ apt-get update
$ apt-get install graylog-server

After installation of the Graylog package is finished, we need to edit the configuration file to set our password in it and we do it using pwgen command 

$ pwgen -N 1 -s 96

 $ echo -n your_password | sha256sum

 

We place the above generated output in the conf file as seen below : 

Assistance reference from https://www.osradar.com/how-to-install-graylog-on-ubuntu-18-04/ - Thanks

Mayan EDMS : Opensource Document Management System : Installation UBUNTU

1.   Mayan EDMS is a web-based free/libre document management system for managing documents within an organization and all functionality is available in its free public version.Mayan EDMS is an electronic vault for your documents. Mayan EDMS has advanced search and categorization capabilities that helps reduce the time to find the information anyone needs. It is free open source and integrates with existing equipment, that means low to no initial investment, and even lower total cost of ownership, reducing operational costs has never been this easy.

2.     This post gives a screenshot reference for installation on a Ubuntu machine.The installation is simple and self explanatory.The list of commands involved too is mentioned below:

 

 

Broadcom Wireless Drivers Installation : Ubuntu[SOLVED]

While one installs a fresh Ubuntu OS in any machine,typically he finds an initial deterrent as the Wifi adapter is not found  installed.This post gives a step by step direction on how and what to do,to get the Wifi working....and this happens since the same is not installed by default.

Since the PCI.ID initials for any Wifi broadcom adapter is prefixed 14e4,thus first we need to find the exact PCI.ID

Type 

lspci -nn -d 14e4:

Once you get the PCI.ID,you just need to ref the following table and find the corresponding driver to be installed.In our case is 14e4:4365 rev 01,so we choos the corresponding bcmwl-kernel-source

PCI.ID              14.04 LTS                        16.04+
------------------------------------------------------------------------------------
14e4:0576           Special Case #1                   UNKNOWN      
14e4:4301           firmware-b43-installer            firmware-b43-installer      
14e4:4306           firmware-b43-installer            firmware-b43-installer      
14e4:4306 rev 02    firmware-b43-installer            firmware-b43-installer      
14e4:4306 rev 03    firmware-b43-installer            firmware-b43-installer      
14e4:4307           firmware-b43-installer            firmware-b43-installer      
14e4:4311           firmware-b43-installer            firmware-b43-installer      
14e4:4311 rev 01    firmware-b43-installer            firmware-b43-installer      
14e4:4312           firmware-b43-installer            firmware-b43-installer      
14e4:4313           firmware-b43-installer            firmware-b43-installer               
14e4:4315           firmware-b43-installer            firmware-b43-installer      
14e4:4315 rev 01    firmware-b43-installer            firmware-b43-installer
14e4:4318           firmware-b43-installer            firmware-b43-installer                
14e4:4318 rev 02    firmware-b43-installer            firmware-b43-installer                
14e4:4319           firmware-b43-installer            firmware-b43-installer               
14e4:4320 rev 02    firmware-b43-installer            firmware-b43-installer               
14e4:4320 rev 03    firmware-b43-installer            firmware-b43-installer            
14e4:4321           firmware-b43-installer            firmware-b43-installer  
14e4:4324           firmware-b43-installer            firmware-b43-installer        
14e4:4325           firmware-b43-installer            firmware-b43-installer      
14e4:4328           firmware-b43-installer            firmware-b43-installer         
14e4:4329           bcmwl-kernel-source               bcmwl-kernel-source        
14e4:432a           bcmwl-kernel-source               bcmwl-kernel-source        
14e4:432b           bcmwl-kernel-source               bcmwl-kernel-source        
14e4:432c           bcmwl-kernel-source               bcmwl-kernel-source        
14e4:432d           bcmwl-kernel-source               bcmwl-kernel-source       
14e4:4331           firmware-b43-installer            firmware-b43-installer          
14e4:4335           firmware-b43-installer            firmware-b43-installer      
14e4:4350           firmware-b43-installer            firmware-b43-installer  
14e4:4353           Special Case #1                   UNKNOWN        
14e4:4353 rev 01    Special Case #1                   UNKNOWN                 
14e4:4357           Special Case #1                   UNKNOWN        
14e4:4358           bcmwl-kernel-source               bcmwl-kernel-source
14e4:4359           bcmwl-kernel-source               bcmwl-kernel-source       
14e4:4360           bcmwl-kernel-source               bcmwl-kernel-source    
14e4:4365           bcmwl-kernel-source               bcmwl-kernel-source      
14e4:4365 rev 01    bcmwl-kernel-source               bcmwl-kernel-source      
14e4:43a0           bcmwl-kernel-source               bcmwl-kernel-source
14e4:43ae rev 02    UNKNOWN                           UNKNOWN     
14e4:43b1           bcmwl-kernel-source               bcmwl-kernel-source        
14e4:43b1 rev 03    bcmwl-kernel-source               bcmwl-kernel-source              
14e4:4727           bcmwl-kernel-source               bcmwl-kernel-source        
14e4:4727 rev 01    bcmwl-kernel-source               bcmwl-kernel-source        
14e4:a962           firmware-b43-installer            firmware-b43-installer

Thats all folks...ready to go

VMware Workstation Player installation @UBUNTU 16.04 LTS

 1.    VMware Workstation Player is a virtualization software package for x64 computers running Microsoft Windows or Linux, supplied free of charge by VMware, Inc. VMware Player can run existing virtual appliances and create its own virtual machines and uses the same virtualization core as VMware Workstation, a similar program with more features, but not free of charge. VMware Player is available for personal non-commercial use, or for distribution or other use by written agreement. VMware, Inc. does not formally support Player, but there is an active community website for discussing and resolving issues, and a knowledge base.

2.   This post brings out few steps with screenshots and source details of download for the VMWare workstation version for Linux. The bundle package can be downloaded at https://my.vmware.com/en/web/vmware/free#desktop_end_user_computing/vmware_workstation_player/12_0

3.   The file I downloaded from the above link is 

VMware-Player-12.1.1-3770994.x86_64.bundle

4.   Few steps to install this file further involves going to the terminal and executing the following commands :

chmod a+x VMware-Player-12.1.1-3770994.x86_64.bundle

the next step

sudo ./VMware-Player-12.1.1-3770994.x86_64.bundle

 this is followed with the following gui shots :

Ready to roll..........