How to measure your Ruby method performance ? [Ruby, Benchmark]

Hey Guys,

Today, we will use our reverse methods that we have built in the previous post and measure how good they are comparing to the Ruby built in reverse method. For this purpose we will use Ruby Benchmark Module.

All we need to do is wrap everything in Benchmark block, and every single method we want to examine in the report block. As you can see from the code below, I am also passing custom headers to differentiate methods.

def time_required_comparison
  Benchmark.bm do |bm|
	bm.report('c-rev-string') do
		reverse_whole_string('Welcome to myprogrammingblog')
	end
	bm.report('s-rev-string') do
		'Welcome to myprogrammingblog'.reverse
	end

	bm.report('c-rev-words') do
		reverse_each_word_in_a_string('Welcome to myprogrammingblog')
	end
	bm.report('s-rev-words') do
		'Welcome to myprogrammingblog'.split(" ").each {|word| word.reverse!}.join(" ")
	end

	bm.report('c-rev-word-order') do
		reverse_word_order_in_a_sentence('Welcome to myprogrammingblog')
	end
	bm.report('s-rev-word-rder') do
		'Welcome to myprogrammingblog'.split.reverse.join(" ")
	end

  end
end


For a full source code please refer here .

The output of this code will look like this:

             user     system      total        real
c-rev-string 0.000000 0.000000 0.000000 ( 0.000009)
s-rev-string 0.000000 0.000000 0.000000 ( 0.000002)
c-rev-words 0.000000 0.000000 0.000000 ( 0.000011)
s-rev-words 0.000000 0.000000 0.000000 ( 0.000004)
c-rev-word-order 0.000000 0.000000 0.000000 ( 0.000004)
s-rev-word-rder 0.000000 0.000000 0.000000 ( 0.000003)


What we really care about is ‘real’ runtime, which shows how long it took to execute this method.

We can see that all of the methods are as British would say ran “bloody” fast. We can also see that, for example ,Ruby’s built in reverse method is much faster than my implementation (If it wasn’t so, I would write my own programming language, right? :) ).

Benchmark Module is very helpful to quickly determine if there is a bottleneck in your implementation, also useful to determine what library to choose based on the real runtimes.

For more advanced profiling, many Ruby developers suggest to use  ruby-prof gem. We will definitely cover it in one of the future posts.

That was it for today,

You have a “Lovely” day. (I am in British mood today :) )

Anatoly

Posted in Ruby | Tagged , , , | Leave a comment

How to reverse a String not using built-in reverse method ? How to reverse every word in a sentence? [Ruby]

Hello again!

I am continuing answering to your questions, this one came from marthab86_.

marthab86_ asks:

“I know Ruby has .reverse! , but can you show me how to actually reverse a word or a sentence without reverse ?”

I am sure that there are many resources that have solution for this problem, but since I am asked, lets have one more example here :)

Here is what we are going to do. We are going to algorithmically and via the code answer to the following questions in Ruby:

a. How to reverse a string  ?
b. How to reverse each word in a sentence?
c. How to reverse word order in a sentence?

Ok, let’s start.

a. How to reverse a string ?

Since I didn’t have any instructions about complexity, I will do it  the easiest way.

a. Get a number of characters of the string we want to reverse
b. Loop that loops through each characters of the string from last to first
d. Have a new string that append characters, one at a time from, the old string to the new string

If we put it in a code, it will look like this:
Note: I am using Ruby, but this will be almost the same in any other language


def reverse_whole_string(string)
	raise StandardError.new('Passed string cannot be reversed because it is empty') if string.nil? || string.empty?
	end_loop = string.length
	new_string = ''
	while end_loop >= 0
		end_loop -= 1 # arrays start with 0, so we need to do end_loop -1 first
		new_string += string[end_loop]
	end
	new_string
end

The only addition I have here is that I have raised StandardError if a String is nil or an empty string, otherwise it is exactly the same as an algorithm above.

b. How to reverse each word in a sentence?

This one is a bit more complex, but the idea is the same:
a.  For easier manipulation transform sentence into an array of words by splitting string by space
b. Loop through each word in a sentence
c. Reverse each word in a sentence the same way we reverse the string in the example above
d. Add reversed word to a new array
e. Convert new array of reversed words into String by joining elements by space

Here is the code:

def reverse_each_word_in_a_string(string)
	raise StandardError.new('Passed string cannot be reversed because it is empty') if string.nil? || string.empty?
	words = string.split(" ") #splitting string into an Array of words
	new_sentence = []
	words.each do |word|
		# doing the same thing as reverse_whole_string
		# but with eah word from an array
		end_loop = word.length
		new_string = ''
		while end_loop > 0
			end_loop -= 1
			new_string += word[end_loop] #Reversing every letter of each word
		end
		new_sentence << new_string # appending every word to an new Array
		# or you could just do this: new_sentence << reverse_whole_string(word)
	end
	new_sentence.join(' ') #joining Array into a string delimited by spaces
end

This is it, not so bad right ?

So the last thing is:

c. How to reverse word order in a sentence?

a. Convert sentence into an array of words by splitting string by space
b. Loop through each  word starting from last
c. Add every element of an array in a reverse order to a new array
d. Convert array into string by joining array by space

def reverse_word_order_in_a_sentence(string)
 raise StandardError.new('Passed string cannot be reversed because it is empty') if string.nil? || string.empty?
 words = string.split(" ")
 new_sentence = []
 end_loop = words.length
 while end_loop > 0
  end_loop -= 1
  new_sentence << words[end_loop]
 end
 new_sentence.join(' ')
end

This is it, if you have any questions or concerns please leave them in the comment section.

Source code  is available here .

In the second part we will talk about, why built-in ruby functions  are better, and how to measure runtime of your application.

Stay tuned,

Anatoly

Posted in Ruby | Tagged , , , , , | 1 Comment

Tutorial: How to get data from external API ? PART 2: Sending Request, Parsing Output [Ruby, JSON, REST]

Here we are with Part 2 of our tutorial. Who missed the first part – it’s here. Also if you missed it please take a look at the Github Movies-CLI app, which I am taking my examples from.

In this part we will discuss two things:

1. Making HTTP request to the API Path
2. Parsing response

I know your time is precious so lets start.

 Making HTTP request to the API Path

By this time you should already have all  your  API configurations loaded into @config of RottenTomatoes::API class, if they are not please check out Part 1 .

To send request to the third party API (RottenTomatoes in our case) we will need to use a gem. I am using  RestClient gem. (Note: You can if you want use net/http built in Ruby functionality, but I find RestClient easier to use).

In regards to gem installation – of course you can manually install gem but I prefer to use a Gemfile.

After installing you will need to require RestClient with:


require 'restclient'

Now all we need to do is put together our configuration into a URL, that RestClient will call.

I am doing this within a separate method:

# Constructs base url used for API calls from api_paths configuraiton file
#
# @return [String] base url
  def base_url
	if @config.has_key?('rottentomatoes')
	  @config['rottentomatoes']['protocol'] + @config['rottentomatoes']['host'] + @config['rottentomatoes']['api_path']
	else
	  raise StandardError, 'api_paths.yml must have rottentomatoes information such as host, protocol and api_path'
	 end
  end

This method is pretty straightforward. It can be enhanced to check if @config has every part of the URL,or you can even create ‘initializer’ that won’t even go to this method if @config is not fully populated with information, but for the purpose of this tutorial I left it straightforward.

So we check if there is ‘rottentomatoes’ key in the @config, and if there is we build URL, otherwise we raise an exception.

Now lets go to the part where we call this URL:

# Receives String, makes call to RottenTomatoes and returns JSON object with movies information from RottenTomatoes
#
# @param [String] title movie title to get more info about
#
# @return [HashMap] JSON with movie information
def get_movie_by_title(title)
# using RestClient make call to restful API with api_ley title and max_movies_per_output
  RestClient.get(base_url,
    {:params =>
      {
        :apikey => @config['rottentomatoes']['api_key'],
        :q => title,
        :page_limit => @config['rottentomatoes']['max_movies_per_output']
      }
  # act on response result
   }) { |response, request, result, &block|
      case response.code
      # if success connection start acting on result
	when 200
	  json_result = JSON.parse(response.body)
	  if json_result.has_key?('movies')
	    put_json_into_movie_obj(json_result['movies'])
	  end
      # if not succesfull just raise an exception
	else
	  raise StandardError, "Cannot connect to RottenTomatoes API, please check config/api_paths.yml for valid api_key"
	end
  }
end

The sole purpose of this method is to call the RottenTomatoes API based on the ‘title’ passed to this method. I am using RestClient.get because current API path expects GET Http request. I am using base_url that we seen earlier and 3 parameters, two of which are taken from the @config and one supplied by user.

So my URL looks the same the an example we seen in RottenTomatoes docs :

 http://api.rottentomatoes.com/api/public/v1.0/movies.json?apikey=[your_api_key]&q=Toy+Story+3&page_limit=1

Parsing response

After sending request via RestClient, I am checking response.code, if it is not 200 which means successful with body, I am raising an exceptions, otherwise I use built in Ruby JSON functionality to parse JSON object I receive back and call my own function

 put_json_into_movie_obj(json_result['movies'])

to put information from JSON into my custom class RottenTomatoes::Movie .

Note: JSON.parse allows to transform response.body which is string into a JSON string which allows access to different its parts (HashMap).

Last Part of this equation is:


private

  # Puts JSON object into a RottenTomatoes::Movie object
  # @param [HashMap] movie_json JSON object with movie information
  #
  # @return [Array] array of RottenTomatoes::Movie objects
  def put_json_into_movie_obj(movie_json)
    movie_json.map { |movie| RottenTomatoes::Movie.new(movie['id'], movie['year'], movie['gernes'],																												 
                                                       movie['title'], movie['release_dates'], movie['ratings'],																											
                                                       movie['synopsis'], movie['posters'], movie['abridged_cast'],																												  
                                                       movie['runtime']) }
  end

This piece of code is super straightforward, what it does it maps and saves different parts of JSON into RottenTomatoes::Movie .

After this we have our RottenTomatoes::Movie, populated with information from RottenTomatoes API, and we can do anything we want with it!

Pretty straightforward right ?

In the next tutorial I will show you how to prettify your command line output and show your movie information.

If you can’t wait just:

1. pull movie-cli app
2. rename config file from ‘.sample’ to ‘.yml’
3. put your api_key’ into config
4. in movie-cli do bundle update
5. run ruby movie_info.rb ‘Unbroken’

That’s it. You want to call another API ? Create new entry into config, get your API Key, modify RottenTomatoes::API class to parse information you want – and you are good to go.

If you have any questions regarding this, put it in the comments or shoot me an email!

Thanks for reading!

Anatoly

Posted in open-source, Ruby | Tagged , , , , , | Leave a comment

Tutorial: How to get data from external API ? PART 1: Getting API Key, Creating and Loading Configuration File [Ruby, YAML, REST]

I have gotten numerous emails with this question. I have decided to build a small app, that is based on the external API.

Why do you need to get data from external API ?

If you are asking this question, maybe you are reading the wrong post. Just kidding! Keep on reading! You need an external API because it allows you to use data from third party sources. Do you want your website to show the weather from WeatherNetwork ? Or game scores ? Or Twitter/Facebook feed ? This all is done through external APIs.

What is the process of getting data from external API?

Process usually involves several steps:

a. Registering with API vendor to get an API Key
b. Getting API paths
c. Making HTTP request to the API Path
d. Parsing response

Pretty straightforward right ?

Lets put it on practice.

Here is what we are going to do:

We will build part of the Movie Command Line Interface App (movie-cli), that will receive a movie  title through command line, and give us back information about the movie.

See Screenshot Below:
movie-cli

For this example I will be using RottenTomatoes API, however we will build our app in a such a way, that  by making couple of small changes, you will be able to put any other API.

Registering with API vendor to get an API Key

To get an API key from RottenTomatoes you will need to register your “project”. Put as precise information as you can, you can always change it later, so don’t worry too much about this.

After filling in the form you will get an email, and after clicking on the link in the email you will be redirected to the page that will have a random string which is your API Key. Save this key, you will need it!

That’s done. Easy right ?

Getting API Paths

To make HTTP request to the API Path you need to know API path. Best source for that is always documentation. After browsing through documentation you will see this URL:

http://api.rottentomatoes.com/api/public/v1.0/movies.json?apikey=%5Byour_api_key%5D&q=Toy+Story+3&page_limit=1

Lets explore parts of this API call:

1. Protocol:  http://
2. Host/Domain: api.rottentomatoes.com
3. API Path: /api/public/v1.0/movies.json
4. Parameters: apikey=[your_api_key]&q=Toy+Story+3&page_limit=1

All this parts are necessary in order for to get information from third party source. In order to make our app extensible we will create a YAML configuration file where we will keep all this info.

It will look like this (api_paths.yml):


rottentomatoes:
  protocol: http://
  host: api.rottentomatoes.com
  api_key: <YOUR API KEY> # Please Replace this with API KEY from http://developer.rottentomatoes.com/
  api_path: /api/public/v1.0/movies.json
  max_movies_per_output: 5

Note: If you are following along in the GitHub,  please rename api_paths.yml.sample file to api_paths.yml . Please remember to put your API-KEY into api_key section.

So now we have configuration file, where we can store all necessary configurations regarding our third party API`s.

Last thing we will do in this part is we are going to load this configuration file into one of our classes.

How to load YAML file into Ruby class ?

Now we are going to create RottenTomatoes::API ruby class that will load this configuration file. Please refer to Github for the file structure.

In constructor of API class we are going to load YAML configuration file:

 def initialize
   # path can be either from root or for debugging from the curent class
   if File.exists?('../../config/api_paths.yml')
	@config = YAML.load(File.open('../../config/api_paths.yml'))
   elsif File.exists?('./config/api_paths.yml')
	@config = YAML.load(File.open('./config/api_paths.yml'))
   else
     raise StandardError, 'Configuration File cannot be found... please make sure you have api_paths.yml'
   end
end

Note:  When dealing with YAML file do not forget to require ‘yaml’.  

This code is pretty straightforward, since I am dealing with relative paths, and  sometimes I run code inside of the class for debugging, I am checking File.exists? two times, once against  path being relative to the root of my app, other relative to the current directory.

If I cannot read/locate my configuration file I raise an exception.

After RottenTomatoes::API is initialized, you are going to have all your API  configurations loaded into a @config , and available to use.

This is it for the Part 1 of our tutorial. Please stay tuned, as Part 2 will be coming up in the next couple of days, and it will deal with actually sending request to RottenTomatoes using RestClient gem. Of course if you cant wait, just grab the whole app from here .

Once again, if you have any questions concerns or recommendations, please let me know.

Have a great day,

Anatoly

Posted in open-source, Programming Languages, Ruby | Tagged , , , | Leave a comment

How to ensure that filename is always unique ? [Ruby]

Hey guys!

Happy New Year! I am glad to be back this year with some interesting news. Thank you for blasting my mailbox with your questions and suggestions! I will do my best to answer all of them as soon as I can!

One of the readers asked me a  question that I  think is very interesting. Moreover, I have faced the same issue recently, and today I want to share it with you.

So topic of today is:

How to ensure filename is always unique ? [Ruby] 

Why is this a problem?

Imagine you are copying a file ‘a.jpg’  in the directory where this file already exist. Some of the systems I know just give you a validation error and ask to rename a file. But it is a good user experience ? I don’t think so.

So what are our options to tackle this issue ?

Option 1. Append some random string to the filename e.g ‘a-asdsds.jpg’

But is it a solution? How random are those strings ? Does this look appealing ?
I think it is a solution, but a pretty ugly one.

Option 2. Append a number to the filename that is incremented with every creation

This solutions is definitely not a new one, it is used by some OS, and I think it might be the easiest and most elegant one. E.g ‘picture.png’ becomes ‘picture-1.png’ and then ‘picture-2.png’.

Any other options you can think of ? Please comment and let me know ;)

Meanwhile we will go with Option 2.

Next question is:

Based on what should we increment this number ?

Some of the solutions I have seen before is based on counting number of files, and then incrementing count by one. This makes sense, but unfortunately this won’t work. ;)

Why?

Lets say I have files ‘a-1.jpg’, ‘a-2.jpg’,a-3.jpg’. Count is 3 here. Now Imagine I have  deleted file ‘a-2.jpg’, and I want to add a new file. So current Count is 2 (a-1 and a-3), so my next filename based on logic should be a-3 (we increment count by 1 -> 2+1), but it is already taken.

Here is the strategy that I think might work:

Lets say we again have 3 files –  ‘a-1.jpg’, ‘a-2.jpg’,a-3.jpg’. We will take the numbers of those files (1,2,3) find the largest and increment it by one. So if we apply to it to previous use-case – we will delete ‘a-2′, but largest number is still 3, so new filename will be ‘a-4′.

What do you think ?

I know that most of you are code-maniacs and want to dig into code right away, so I will shut up now and show you some code I have built to do exactly this :)


 # Function generates uniq file name from the String passed to it
  # based on extension and basename
  #
  # @param [String] new_file_name desired file_name
  #
  # @return [String] generated file name
  def self.generate_random_filename(filename)
    ext = File.extname(filename)
    name = File.basename(filename, ext)
    related_file_indexes = []
    @file_list.select  do |file|
      if File.basename(file).include?(name) && File.extname(file) == ext
        related_file_indexes << file.split("-").last.to_i
      end
    end
    return name +  '-' + (related_file_indexes.max + 1).to_s + ext
  end

Short Explanation:

1. I take the file lets say ‘a.jpg’, and divide it into ‘a’ as a name and ‘.jpg’ as an extension.
2. I have a @file_list array that is pre-populated with filenames in the directory lets say @file_list is  [‘a-1.jpg’,’a-3.jpg’, ‘a.jpg’,’a-1.png’] .
3. I check if filename in @file_list consist of ‘a’ and with extension ‘.jpg’, thus I get 3 files [‘a-1.jpg’,’a.jpg’,’a-3.jpg’]
4. I have another array ‘related_file_indexes’, that splits every element of an array by last ‘-‘ and returns the number which gives me [‘1′,’3′]
5. I built filename with name, max of 1 and 3 which is 3+1=4 and append extension so my output will be ‘a-4.jpg

Full version of this code  with unit tests can be found here .

Once again, all suggestions and recommendation are more than welcome in the comments to this post ;)

Have a wonderful day!

Anatoly

Posted in Ruby, Useful Functions | Tagged , , , , , | 2 Comments

How to mock file upload with RSpec 3.0 ? [SOLVED]

One more note on RSpec 3.0, this time one useful function that mocks File Upload , saves file/archive content to memory so after it you can do whatever your soul wants with it. Test file/archive should exist in the file system.


def mock_archive_upload(archive_path, type)
  return ActionDispatch::Http::UploadedFile.new(:tempfile => File.new(Rails.root + archive_path , :type => type, :filename => File.basename(File.new(Rails.root + archive_path))))
end

 

e.g of use:


#saves archive into memory so it can be manipulated in tests
mock_archive_upload("../fixtues/my_archive.zip", "application/zip")

Posted in open-source, Programming Languages, Rails, Rspec, Ruby | Tagged , , , , , , | Leave a comment

How to stub File.read in RSpec 3 [RAILS, TDD]?

 

Recently, I have been doing whole bunch of spec rewriting from RSpec 2 to Rspec 3. As a result I have found that documentation is not that straightforward, and some cases you need to really spend time and figure out yourself. So I’ll be posting some cases that I have not been able to find in docs, just as notes for myself, and hopefully it will be useful for someone else.

 

Use Case:

I have JSON file that I need to read, and I already have tests that this JSON file opens properly.  I also have a Validator that looks for certain patterns keys in this JSON file. Validator leaves separately from the main app, and I don’t want to replicate File Upload -> File Save to Cloud -> File Open in my Validator Tests, I just want on every File.read to return certain data, and it does not matter to me what exactly is being read, and if path exists or not.

How would I do it in Rspec ?

data = {:name => "test" , :description => "test"}.to_json
allow(File).to receive(:read).and_return(data)

What does it give you ?

It looks for every File.read in the scope of your spec and instead of actually reading file returns ‘data‘ every time.

Cool, huh ?

:)

Anatoly

Posted in Programming Languages, Rails, Rspec | Tagged , , , , , | Leave a comment