Remove all the records from ets table having datestamp older than 10 seconds

Question

I have an ets set table in an elixir app. I need to clean-up records which have their updated_at field older than 10 seconds. Is there a way I can set expiry or do it manually without iterating over all the records? I match records based on the timestamps greater than given time.

example record:

key: key_1
record: %{id: key_1, updated_at: ~N[2018-12-19 10:08:47.803075]}

so far I have this code

def clean_stale(previous_key) do
  if previous_key == :"$end_of_table" do
    :ok
  else
    device = get(previous_key)
    next_key = :ets.next(__MODULE__, previous_key)
    if NaiveDateTime.diff(NaiveDateTime.utc_now, device.last_recorded_at) > 10 do
      remove(device.id)
    end
    clean_stale(next_key)
  end
end

Answer 1

If you store the "updated at" time as an integer instead of as a NaiveDateTime struct, you can use a match spec.

For example, to get the current time as the number of seconds since the Unix epoch:

> DateTime.to_unix(DateTime.utc_now())
1545215338

You can do something like this:

iex(3)> :ets.new(:foo, [:public, :named_table])
:foo
iex(4)> :ets.insert(:foo, {:key1, DateTime.to_unix(DateTime.utc_now())})
true
iex(5)> :ets.insert(:foo, {:key2, DateTime.to_unix(DateTime.utc_now())})
true
iex(6)> :ets.tab2list(:foo)
[key2: 1545215144, key1: 1545215140]
iex(7)> :ets.select_delete(:foo, [{{:_, :"$1"}, [{:<, :"$1", 1545215144}], [true]}])
1
iex(8)> :ets.tab2list(:foo)
[key2: 1545215144]

In the call to ets:select_delete/2, I pass a match specification. It consists of three parts:

• With {:_, :"$1"}, I perform a match on the records in the table. In this example, I have a tuple with two elements. I ignore the key with :_, and assign the timestamp to a match variable with :"$1".
• With [{:<, :"$1", 1545215144}], I specify that I only want to match records with a timestamp before this time. In your case, you would calculate the time ten seconds in the past and put that value here.
• With [true], I specify that I want to return true for matching records, which in the case of select_delete means "delete this record".

So after calling select_delete, only the second record remains in the table.

---

If the timestamp is inside a map, you can use map_get to access it and compare it:

:ets.select_delete(:foo, [{{:_, :"$1"},
                           [{:<, {:map_get, :updated_at, :"$1"}, 1545215339}],
                           [true]}])

Or (in Erlang/OTP 18.0 and later) match out the map value:

:ets.select_delete(:foo, [{{:_, #{updated_at: :"$1"}},
                           [{:<, :"$1", 1545215339}],
                           [true]}])