How should I import this data into my database?
I have database with thousands of records
Code  | Name  | Price
00106 | Water | 9.99
00107 | Onion | 8.99
Which is coded in GES file like below:
00F means column header
00I means insert a row
there are also others like(00D for delete row or 00U for update)
00F
0101
02Code
031
00F
0102
02Name
031
00F
0103
02Price
030
00I
0100106
02Water
030999
00I
0100107
02Onion
030899
I want to create importer which process this file and push it into my
database. So I started implemented that:
class Importer
  CONN = ActiveRecord::Base.connection
  F = "00F"
  I = "00I"
  def extract_to_database(collection)
    add       = true
    tmp       = []
    type      = F
    inserts   = []
    collection.each_with_index do |line, i|
      _type    = line.strip
      _changed = [F,I].include? _type
      if _changed && i > 0
        case type
        when F then @f << tmp
        when I
          inserts.push "(#{tmp[1]},'#{tmp[2]}','#{tmp[3]}')"
        end
        tmp  = []
        type = _type
      end
      tmp << line
    end
    sql = "INSERT INTO products (`code`, `name`, `price`) VALUES
#{inserts.join(", ")}"
    CONN.execute sql
  end
end
There is one problem with that, I would like to refactor that using
functional programming.
And I will have to find other model by code and put to products table
related some_model_id so this can complicate whole process.
 
No comments:
Post a Comment