AWS Developer Associate - 2

S3 - Simple Storage Service

1. for files/images/web pages
2. Fundamentals
1. Key
2. Value
3. Version
4. Metadata
5. Subresources - bucket specific configuration
1. Policies, ACL
2. CORS - Cross Origin Resource Sharing
3. Transfer Acceleration
3. object based, not block storage
4. HA, DR
5. file size: 0bytes to 5 TB
6. Unlimited Storage
7. HTTP 200 on successful upload
8. Data Consistency
1. Read after Write for PUTS of new objects
2. Eventual consistency for overwrite PUTS
9. Amazon guarantees 99.9% availability
10. Amazon guarantees 99.(11 9s) % durability
11. Tiers/Classes
1. Regular - can sustain the loss of 2 facilities concurrently
2. Infrequent Access - low fee, but charged for retrieval
3. OneZone - 99.5% availability, 20% less cost than regular
4. Reduced Redundancy Storage: 99.99% durability, ex for thumbnails, may be deprecated
5. Glacier: Cheap, optimized for infrequent access, takes 3-5 hours to restore data, archives
6. Intelligent Tiering - unknown access patterns, data not accessed for 30+ days, moved to infrequent, after access, moved to frequent tier, optimized cost
1. Frequent
2. Infrequent
12. Charged for:
1. Storage/GB
2. Requests
3. Storage mgmt - inventory/analytics/tags
4. Data mgmt - data taken out of S3
5. Transfer acceleration

AWS Developer Associate - 1

Amazon does not launch new services in all regions.
Normally services are first launched in US-East i.e. North Virginia region

Identity and Access Management

In here you get a custom link for your login(contains random numbers), which you can customize to any unused word you want.

Minimum security that AWS recommends:

1. Delete root access keys
2. Enable MFA
3. Create IAM users
1. Root account should not be used for anything
2. Restrict access to users and give them access to what they need
4. Use Groups to assign permissions
1. We can also give permissions directly to users
5. Apply password policy

User creation:
You get username and password for console login and access key and secret access  key for API


EC2

Elastic compute cloud
Provides resizable compute capacity
Payment:

1.  On Demand - fixed rate by hour (linux by second) - no commitment, no upfront payment, good for learning
2.  Reserved - commitment for 1 or 3 year, discounted hourly rate, 
1. Standard RI: cost can be 75% off on demand
2. Convertible RI: 
3. Scheduled RI:
3. Spot - allows you to bid, good for flexible timings, price goes above bid, AWS will terminate and not charge, if you terminate, will be charged
4. Dedicted hosts - usefull if you have server bound software licenses, regulatory requirement for no multi-tenancy

EC2 Instance types:
FIGHTDRMCPX

EBS

virtual disk
elastic block storage - attach to EC2
is replicated

EBS Types:

General purpose SSD: 10K IOPS burst upto 30K IOPS
Provisioned IOPS SSD: DB, extreme performance, >10K IOPS

Throughput optimized HDD (ST1) : no root volumne, data warehouseing, log processing
Cold HDD (SC1): file server, lowest cost, bootable

To connect to EC2, we use SSH for linux and RDP for windows

Load Balancers: Application(layer 7)/Network(most expensive/perf)/Classic(layer 7 or 4)

504 error is gateway timeout, app did not respond

Use header X-Forwarded-For to find out who the load balancer forwarded ipv4

Route53: DNS service

AWS CLI

aws configure
aws s3 ls
aws s3 mb s3://bucket
aws s3 cp hello.txt s3://bucket

User with CLI access might not need console access, dont give it
Always create groups
Secret access key will be shown only once

Dont use access key, use roles instead

RDS

ElastiCache : in-memory cache, faster than DB
Supported engines:

1. Memcached
2. Redis

Backup

1. Automated
   1-35 days recovery, stored on S3, free
2. Snapshot
   manually, stored after RDS is deleted

Restoring - is a new RDS instance

Multi AZ - disaster recovery, sync

Read replicas - improve performance, async replication, scaling

for loop in Scala

In Scala, the for is actually foreach, you write it as follows:

val filesHere = (new java.io.File(".")).listFiles
for (file < - filesHere)
println(file)

for (i < - 1 to 4) //gives 1,2,3,4
for (i < - 1 until 4) //gives 1,2,3

We can not only iterate over a sequence but also filter it as follows:

for (
file < - fileshere
if file.isFile;
if file.getName.endsWith(".scala")
) println(file)

Nested loop:

for (
file < - fileshere
if file.getName.endsWith(".scala");
line < - fileLines(file)
if line.trim.matches(pattern)
) println(file +": "+ line.trim)

Creating new collection:

def scalaFiles =
for {
file < - fileshere>
if file.getName.endsWith(".scala")
} yield file

Immutable objects

Positives:

1. State does not change overtime
2. Can pass them around freely
3. Threads don't share an object 
4. Can identify an object by its hash

Negatives:

1. Large objects result in performance hit when they are duplicated

Currying

After a lot of research I kinda have a sense of why we need currying in functional languages - it turns out, there are some (don't know which) analytical techniques which can only be applied to functions with single argument.

So if we want to use these (mysterious) techniques, we need currying :)

Oh, and of course it gives us a different style of reuse

while ((line = readLine()) != "") doesn't work in Scala

In C++ or Java, line = readLine() would return the line itself, however, in Scala, assignment to var type variable returns () i.e. Unit, so it cannot be compared to "" in a while loop.

Pure functional languages do not have loops because of this same reason, it does not return anything

Scala Class and Object

class checksumAccumulator {

private var sum = 0

def add(b: Byte): Unit = {
sum += b
}

def checksum(): Int = {
~(sum & 0xFF) + 1
}

}


By default sum is public, we have to explicitly make it private.
Parameter b is by default val, so we cannot re-initialize it in checksum method, there is no return keyword, in such case, last value computed by method is returned - this is the recommended style - avoid explicit and multiple return statements

class ChecksumAccumulator {
private var sum = 0
def add(b: Byte) { sum += b }
def checksum(): Int = ~(sum & 0xFF) + 1
}

since our methods are just single statements, they can be written without {}, also when method does not return anything, i.e. its return type is Unit, we can skip writing the return type and "=", like in the add method

all datatypes can get converted to Unit - it simply looses the data
semicolon at the end of the statement are optional

object ChecksumAccumulator {
}

defining object instead of class gives singleton of the class called companion object class
A class and its companion object can access each other’s private members.

To run a Scala program, you must supply the name of a standalone singleton object with a main method that takes one parameter, an Array[String], and has a result type of Unit. Any singleton object with a main method of the proper signature can be used as the entry point into an application.

object Summer {
def main(args: Array[String]) {
}
}

We can also write the following:

object FallWinterSpringSummer extends Application {
}

Who uses functional programming

• Erlang
• Amazon
• Yahoo
• Facebook
• Whatsapp
• Haskell
• AT&T
• Google
• Facebook
• NVIDIA
• Banks - ABN Amro, BoA, Barclay, Credit Suisse, Deutche Bank, SC
• OCaml
• Facebook
• Bloomberg
• Scala
• LinkedIn
• Twitter
• Sony
• Seimens
• Clojure
• Facebook
• Walmart
• Citi

Why functional programming matters now

• Clock speed of processors are not increasing
• Cores on a processor are increasing
• In future we may have hundreds of cores
• Execution is no longer the bottleneck
• Memory is a bottleneck
• Foo(f(x), f(x))
• both f(x) might get executed on different cores
• it will not matter if there is no state change
• Hence functional programming

Functional Programming principles

Functional programming has 2 principles:

• Functions are first class values
• pass them as parameter
• return them from a function
• store them in variable
• define function inside a function
• have function literals in the code - like int lieral 42 => anon function
• Operations should map input values to output
• Not change data in place - like what OO does
• Makes data immutable
• No side-effects => referentially transparent

Attended conference on functional programming

Attended a conference (http://functionalconf.com/) on functional programming.

It gave me the push needed to take up learning a new language. I have started exploring SCALA, I will be posing my learning here.

I am the one in white shirt

My other blogs

http://mongo4data.blogspot.in/ - I have decided to give a lot of attention to MongoDB, this new blog will have posts about my experience and experiments with it.

SDD 2014

SDD 2014 held during 19-23 May 2014

SDD (Software Design & Development) is an annual conference for software designers, developers and architects, which takes place at the Barbican Centre in London.

That's me at watching something during one of the breaks

Saving TFS credentials

Go to Control Panel\User Accounts\Credential Manager

Add windows credentials - put network address as your TFS server and your credentials.

That did it. VS2008 on windows server 2008 R2 did not ask for TFS credentials.

This thing also worked on command prompt too - I didnt have to provide credentials for "tf get"

For windows XP :

Go to Control Panel\User Accounts\Advanced\Manage Password - Add

If your machine does not show Add button , try this

1. Enable your "Protected Storage" service from services.msc

2. run secpol.msc Go to Local Policy -> Security Options -> Disable the policy "Network access: Do not allow storage of credentials or .NET Passports for network authentication"

Using Magnifier in presentations

This is a must know for anyone who gives presentations.

Windows 7 has an awesome tool - magnifier (magnify.exe)

Heres how you can use it :

• Windows logo key + Plus to zoom in
• Windows logo key + Minus to zoom out, and
• Windows logo key + ESC to exit.

Also, if you are showing code in Visual Studio, all you have to do is Ctrl + mouse scroll to increase or decrease the text size.

Debugging Blogspot+Silverlight+Dropbox issue

I traced my browser calls when it loaded the demo silverlight app and I got the following:

Request URL:http://dl.dropbox.com/u/4186718/Nitin.Spike.XAPHosting.xap
Request Method:GET
Status Code:200 OK
Response Headers
Content-Type:application/octet-stream
Date:Sun, 19 Sep 2010 06:12:06 GMT
Server:dbws
accept-ranges:bytes
cache-control:no-cache
content-length:4379
etag:10n
pragma:no-cache
x-robots-tag:noindex,nofollow

The problem is that the Content-Type is application/octet-stream, it should have been application/x-silverlight-app. Okay so we know the problem, how do we fix it? I am using dropbox, are they suppose to push with correct content-type or do i need to configure it somewhere... I have asked this question in their forums...

Testing Silverlight

Okay it doesnt work, but i see a lot of people displaying silverlight content on blogspot. I wonder whats going wrong. I guess i am done using these free blog hosting sites... its time to move to my own blog - its expensive but i will get to control a lot of stuff.

Update: I have finally hosted my silverlight XAP on a paid host and it works. :)

Enum extensions

Sharing these Enum extension methods I wrote yesterday.

using System;
using System.ComponentModel;

namespace Nitin.Spike.EnumExtensions
{
    class Program
    {
        static void Main()
        {
            const Colors someColor = Colors.Red;
            string description = someColor.Description();
            if (someColor.HasDescription())
            {
                if (someColor.HasDescription("indicates stop", StringComparison.OrdinalIgnoreCase))
                {
                    Console.WriteLine("Works");
                }
            }
        }
    }
    public enum Colors
    {
        [Description("Indicates Stop")]
        Red,
        [Description("Indicates Nothing")]
        Blue,
        [Description("Indicates Go")]
        Green
    }
    public static class EnumExtensions
    {
        public static string Description(this Enum someEnum)
        {
            var memInfo = someEnum.GetType().GetMember(someEnum.ToString());

            if (memInfo != null && memInfo.Length > 0)
            {
                object[] attrs = memInfo[0].GetCustomAttributes(typeof(DescriptionAttribute),false);

                if (attrs != null && attrs.Length > 0)
                    return ((DescriptionAttribute)attrs[0]).Description;
            }
            return someEnum.ToString();
        }
        public static bool HasDescription(this Enum someEnum)
        {
            return !string.IsNullOrWhiteSpace(someEnum.Description());
        }
        public static bool HasDescription(this Enum someEnum, string expectedDescription)
        {
            return someEnum.Description().Equals(expectedDescription);
        }
        public static bool HasDescription(this Enum someEnum, string expectedDescription, StringComparison comparisionType)
        {
            return someEnum.Description().Equals(expectedDescription, comparisionType);
        }
    }
}

High Speed Internet

The first time I connected to internet was some 9 years back, I got my own internet connection around 8 years back, and at that point of time I used to get 56 Kbps speed, and today I have a 512 Kbps line.

Is this speed enough for me? Hell no.. why not? because even today I can’t watch my videos directly off the web. Although I can watch low quality videos online, but now I want HD quality. I wonder if I will ever be satisfied with my connection.

WPF Converter values

I posted this at stackoverflow : http://stackoverflow.com/questions/3204422/wpf-corelating-multibindings-and-converters-values

The problem was how do you handle indexes when your converter value array is of large size. I thought over it and come up with this:

https://www.assembla.com/code/TechnologyAndMe/subversion/nodes/ConverterValues
(Download complete source using a SVN client from http://subversion.assembla.com/svn/TechnologyAndMe/ConverterValues/)

So now all you have to do is :

Here Prop1,Prop2,Prop3 are the actual values that you want to pass to the converter, Value1, Value2, Value3 are just ids for Properties below them. Now in the converter, just use my BoundValues class and accessing the properties become as easy as saying :

public object Convert(object[] values, Type targetType, object parameter, CultureInfo culture)
{
 var boundValues = new BoundValues(values);

 return string.Format("{0} {1} {2}", 
  boundValues["Value1"], boundValues["Value2"], boundValues["Value3"]);
}

Where BoundValues["Value1"] returns value of Prop1, so now I no longer have to worry about managing the sequence of bindings in my XAML.