Arc Forumnew | comments | leaders | submitlogin
MongoDB support in arc?
2 points by shader 3880 days ago | 13 comments
There seems to be a mongo driver for racket: http://planet.racket-lang.org/package-source/jaymccarthy/mongodb.plt/1/0/planet-docs/mongodb/index.html

However, this driver seems to use things like #:keywords that would be incompatible with arc. How would you suggest creating a mongo driver for arc?



2 points by shader 3880 days ago | link

So, I had one idea for supporting #:symbols

Would anyone mind if I changed ssyntax so that ':symbol expanded into the same thing as #:symbol? ":" currently doesn't mean very much as the first character of a symbol. In fact, I think that currently ':symbol == 'symbol.

-----

2 points by shader 3880 days ago | link

After hacking on this for a bit, I determined that adding ((keyword? s) s) to the primary ac cond was necessary either way. So, after adding that the #:symbol syntax works as well, since we're mostly just using the racket reader.

Still, I think that the :symbol syntax is a little cleaner and also matches common lisp. Any reason I shouldn't add it?

-----

3 points by rocketnia 3879 days ago | link

"After hacking on this for a bit, I determined that adding ((keyword? s) s) to the primary ac cond was necessary either way."

That's not necessary either. If you want an Arc expression that evaluates to a keyword, you can already just do this:

  arc> '#:foo
  #:foo
Even so, getting first-class keywords probably won't help as much as you might like:

(Example output from http://tryarc.org/.)

  arc> (= k-idfn ($:lambda (#:k v) v))
  #<procedure:zz>
  arc> (k-idfn '#:k "hello")
  Error: "struct procedure:zz: expects 0 arguments plus an argument with keyword #:k, given 2: #:k "hello""
This function expects 0 positional arguments and a #:k keyword argument. However, we've passed it 2 positional arguments, one of which is #:k and one of which is "hello". We haven't passed it any keyword arguments at all, let alone one with the keyword #:k.

We pretty much have two ways to call this function successfully.[1] First, we can write the function call itself as Racket code, taking advantage of Racket's function call syntax compiler to parse the keywords:

  arc> (let call-k ($:lambda (f v) (f #:k v)) (call-k k-idfn "hello"))
  "hello"
Second, we can build the keyword mapping in a first-class way and use Racket's 'keyword-apply:

  arc> ($.keyword-apply k-idfn ($.list '#:k) ($.list "hello") ($.list))
  "hello"
(Here I've used ($.list ...) as a way to construct ()-terminated instead of nil-terminated lists.)

Note that keyword-apply requires the keywords to be given in a specific order based on their UTF-8 bytes.

If we do a lot of this in the code, I'm thinking we could benefit from one of three things:

- An Arc function that takes an Arc table like (obj k (list "hello")) and calls Racket's 'keyword-apply it to the sorted keywords it expects.

- An Arc macro that looks like a function call, e.g. ($kw-call k-ifn #:k "hello"), but finds any occurrences of keywords and generates the appropriate Racket function call.

- A patch to Arc's own function call syntax so that it parses Racket-style keyword arguments.

I think these solutions would be more effective than shaving one or two heiroglyphics off of '#:k.

---

Not even Racket allows #:k as an expression by itself. It has to be written '#:k. After all, even if #:k evaluated to itself, (list #:k "hello") would not give you a list containing #:k and "hello", since it would try to pass a keyword argument instead.

You might wonder why Racket chose a design with such hoops to jump through. Well, I can't speak for them, but personally I'd give several shallow and hand-wavy reasons:

- It aligns with a certain mental model where positional args are an ordered list and keyword args are an orderless map. (This is the mental model I learned a long time ago in Groovy, so I like it.)

- It lets Racket treat (foo #:a 1 #:b #:c 3) as a compile-time error.

- It lets Racket treat missing/extra keyword args and missing/extra positional args as the same kind of error.

- It means keyword args don't have to be described in terms of positional args. It would be possible to design a lisp that only has Racket-style keyword args, with no positional args whatsoever.

- It means when a programmer wants to get a procedure's arity information dynamically, the result can include information about the keyword args it supports, rather than just the positional args it supports.

---

[1] Hmm, maybe there's a third way to call it successfully:

  arc> (= successfully ($:lambda (#:k v) v))

-----

2 points by shader 3879 days ago | link

My original goal was not to shave characters off of '#:key. I apparently misunderstood how they were supposed to work, and thought that arc wasn't compatible with them. I ended up doing the cosmetic change as part of my process of figuring out how to hack it onto arc.

If what you're saying about '#:key vs #:key is true though, then my test cases were incorrect and I was trying to make something work that shouldn't have.

So, what's the right way to hack on arc to support calling racket functions with keyword args? Or would it be better to make a more arc-idiomatic mongo driver, and how?

-----

2 points by rocketnia 3878 days ago | link

Oops, I think you've been seeing something I wasn't seeing.

It turns out I'm getting much different results on a local setup than I was getting in tryarc.org. Since Arc 3.1 and Anarki use (require mzscheme), all sorts of things are replaced with doppelgangers that doesn't support keyword args, including the function application syntax #%app. In fact the lack of keyword arguments is one of the only things (require mzscheme) is good for. This is something tryarc.org changes, apparently.

MzScheme docs: http://docs.racket-lang.org/mzscheme/index.html

---

"So, what's the right way to hack on arc to support calling racket functions with keyword args?"

I wouldn't say it's "the right way," but we could put the functionality of Racket's 'keyword-apply in a function that takes an Arc table, and then we'd pretty much never need to worry about keywords other than that.

I have an implementation for this, which I should be able to commit with some unit tests now that I know what's going on with MzScheme.

...Actually it might take me a few days to get around to that, so here's the code if anyone wants to use it right away:

  (def $kw-apply (func kwargs . posargs)
    
    ; Convert the last element of posargs (the first-class list) into a
    ; Racket list. If posargs is completely empty, pretend the last
    ; element is an empty list just like Arc's 'apply already does.
    (zap [rev:aif rev._
           (cons (apply $.list car.it) cdr.it)
           (list:$.list)]
         posargs)
    
    (let (ks vs)
         (apply map list  ; Transpose.
           (sort (compare $.keyword<? !0)
             (map [list ($.string->keyword:+ "" (or _.0 "nil")) _.1.0]
               tablist.kwargs)))
      (apply $.keyword-apply func (apply $.list ks) (apply $.list vs)
        posargs)))
Use it like so:

  ($kw-apply my-func
    
    ; keyword args
    (obj a list.1 b list.2 message (list "hello"))
    
    ; positional args
    1 2 3 (list 4 5))
I'm using a table of singleton lists so that we can pass the symbol 'nil as an argument value.

---

"Or would it be better to make a more arc-idiomatic mongo driver, and how?"

I actually have some opinion about "the best way" for this. :)

I think Arc-idiomatic approaches serve no particular purpose, since Arc is a tool for general-purpose computation.[1] I would want a database driver to be idiomatic only for the database itself, so that it serves the more specific purpose of storing data. This can then be accompanied with sugary helper utilities, as long as they're optional.

It seems many ORMs want to bake the sugar into the interface, or they make sugar that has tiny escape hatches for poking at the underlying interface. If sugar is the only thing a programmer (usually) sees, I look at it as though it's a full-on database design of its own... which is rarely favorable since it usually inherits most of the complexity and obligations of the original.

[1] Well, while Arc is a general-purpose tool, it's specifically a language, so Arc-idiomatic approaches serve the particular purpose of making features more accessible to language users.

-----

3 points by shader 3878 days ago | link

Do we actually need the mzscheme dependency? Any reason we couldn't switch to a full racket base?

---

Interesting. I'll have to play around with that. And study it a bit more to figure out how it works. If I understand your description correctly, the reason you're using lists is so that 'nil is interpreted as no value, while '(nil) is interpreted as intentionally passing the value 'nil? How does the function know the difference?

---

Interesting opinion. Unless I misunderstood you, I would have felt the opposite way. Normally, I would expect the purpose of the data interaction layer to be separating the implementation details from the code, so that if changes need to be made to what backend storage system you can just trade it out.

Maybe that is what you mean though, and that's supposed to be a distinction between the "driver" and an additional abstraction layer. Though your comment about ORMs including sugar confuses that a bit. I don't really want sugar per se, just abstraction away from how I'm actually storing the data, within reason anyway.

-----

2 points by rocketnia 3878 days ago | link

"Do we actually need the mzscheme dependency? Any reason we couldn't switch to a full racket base?"

There's always a reason, but these days Anarki has broke all my code enough times that that shouldn't be a concern. :)

I think this would be a positive change.

---

"...the reason you're using lists is so that 'nil is interpreted as no value, while '(nil) is interpreted as intentionally passing the value 'nil? How does the function know the difference?"

Arc doesn't support nil as an element of a table. Setting a table entry to nil removes it. Therefore '$kw-apply will only see non-nil values anyway.

As pg says: "In situations where the values you're storing might be nil, you just enclose all the values in lists." http://arclanguage.org/item?id=493

When I was using Arc heavily, I defined a utility (sobj ...) that was just like (obj ...) but it wrapped everything in a singleton list. That may have been the only extra utility I needed.

I could write (each (k (v)) tab ...) in place of (each (k v) tab ...). I could write tab!key.0 in place of tab!key. I could write (iflet (v) tab!key ...) in place of (iflet v tab!key ...).

It was surprisingly unintrusive, even pleasant.

---

"Normally, I would expect the purpose of the data interaction layer to be separating the implementation details from the code, so that if changes need to be made to what backend storage system you can just trade it out."

I like the sound of that, but I think it's always a bit leaky, unless the result is a full database design that lets people happily forget there's another database underneath.

-----

1 point by akkartik 3879 days ago | link

Oh, is the mongodb library using keyword args? I'm used to lisp where keyword syms can exist outside lambda lists.

-----

2 points by shader 3879 days ago | link

It seems to, though much of that is the ODBC layer, which I'm not sure I like anyway. It generates some constructors and getters and setters that use keywords, and I don't think that fits particularly well with the arc style.

Unfortunately, some of the database options are also keyword based, but maybe wrappers could be made for the few cases that matter? Sadly, I am also a beginner with mongo, so I don't really know what I need or how to do it.

-----

1 point by akkartik 3880 days ago | link

On tryarc (so not anarki):

  arc> ':abc
  :abc
But yeah, feel free. I've often wanted :foo to eval to itself just like in common lisp. Currently I use 'else where I would use :else in wart.

-----

2 points by shader 3880 days ago | link

Interesting. Well, I can push what I currently have, and if necessary remove it later.

Also, one issue is that after adding ssyntax support, :symbol is printed as #:symbol, since they're the same. Should I be adding a special case for that?

-----

1 point by akkartik 3880 days ago | link

Thanks! Looks great. I made one tweak: https://github.com/arclanguage/anarki/commit/c416f3a36f. And I've migrated 'else to :else. I can live with printing keywords in racket syntax. No need to add complexity for something so cosmetic. (Analogous to my thinking on nil: http://www.arclanguage.org/item?id=11864; http://arclanguage.org/item?id=12661; http://arclanguage.org/item?id=12814)

-----

1 point by akkartik 3880 days ago | link

Maybe just expose it at the racket level using xdef?

-----