-
Notifications
You must be signed in to change notification settings - Fork 231
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Integrate with esp8266/Arduino #1
Comments
This should be able to fit fairly well, but there's a lot I don't know. Only thing I'm kind worried about is the way my reflasher works is it uses a scratchpad, verifies, and then flashes itself. If something goes wrong or you break the USB interrupt, you would have to go back to serial flashing to get the thing recovered. Another big recommendation is I know people would probably want to turn this into a CDC device, but I really believe using control messages is the best way to stage data back and forth. It sidesteps a lot of problems with endpoints and keeps things really organized. P.S. Right now my focus is on two other aspects of the project. (1) Demoizing the project and (2) Trying to get USB 1.1 Full Speed working. I still don't know if #2 is possible, but I am going to keep my focus there. I also can continuously work to shrink my code size. Who all knows much about how to set up bootloaders, etc? |
Just realized igrr (https://github.com/igrr) the main contributor of esp8266/Arduino is sitting at @espressif. He probably can help you with your Full Speed ambitions (interrupt/dma issues and what not) |
I just made a post on their forums (though it looks like posts have to be approved) How would I go about requesting this from him? Should I be requesting it of him directly? I know RichardS said he's already on trying to get that info, too. |
Hmm, I don't know how to get the info you need the best. It was just an observation.. |
Hmm, right now things are a bit stalled before we can COMPLETELY rule out full-speed... at least on my end, but I'm not sure what needs to happen arduino-end to start support of this project. |
Is asking @igrr for information still a bottleneck? In my experience, he is very approachable, helpful and friendly and is online on https://gitter.im/esp8266/Arduino most of the time. |
The main thing which needs to happen on Arduino end is making decisions :) Regarding the espusb code itself, I think it would be nice to package all usb-related code into a self-contained module (static library) with a well-defined API. I can understand that some hacking and exploration needs to happen first, but once they are finished some cleanup and refactoring would be much welcome. That would make it easier for others to use this project within other applications/frameworks as is, for example by adding it as a git submodule. Regarding additional info about hardware features of the 8266, i think Angus has already given a series of replies on the 8266 community forum. If there's anything else you want to know, please ask. |
Thanks @igrr for chiming in. I am by no means an expert so in the end it is @cnlohr and @igrr making the decisions, but since I started this thread, I would like to offer my perspective from an user's point of view. Why do people love Arduino? Because it gets us producive extremely fast, without having to worry too much about the low-level infrastructure. Hence, I see 2 main use cases for espusb in esp8266/Arduino:
Whereas it would be elegant to have the code for 1 and 2 combined in just one place, I wouldn't worry to much about having 2 separate copies of the USB code (one in the core/bootloader, one included into the sketch), since with 4 MB of flash, who cares. |
@igrr I wish I had an opinion on the subject. I really don't know what's best, either, regarding where the bootloader should be, and the timeout, etc. I would be concerned to let it boot too quickly, since it may take a bit for the device to enumerate and a PC task to catch it if you want it to stick around in the bootloader. I tried finding some spec for usb bootloading and didn't find one. I still much prefer use of control messages over just about anything else. I am going to continue refining the main espusb project, now that I am putting the full-speed stuff on hold, indefinitely (unless espressif comes back with something). I have some updates just committed in 90e8a22. I will be splitting the code apart so the per-device stuff is separate from usb.c. Right now, endpoints are handled externally, but descriptors and custom control messages are in there. Most of this can be done in parallel - both work on Arduino and my work. Especially if it results in specific requests for changes to my code. P.S. It's currently ~8.1 seconds to flash to the scratchpad via usb and another 2 seconds to copy it over and reboot. All that said... I am not a fan of CDC devices. And, I don't even know if they're supported on all OSes in low-speed mode. One other thing that is convenient. I do have code now that causes the bus to re-enumerate. |
I am getting very close to what I want for the "library" Right now, usb.h, usb.c, usb_asm_1bit.S and usb_table_1bit.h are all librariezed. All of the descriptors, custom behavior, etc. is located elsewhere. Current usage: Total SRAM: 232 bytes + Descriptors, 317 bytes (Could be stored elsewhere) If only a bare minimum USB device is needed, could be smaller yet, but not by too much. |
Ping @igrr your input is appreciated. |
For the bootloader mode, this amount of RAM is not an issue, while for the application the amount of iram required may be a problem. We don't have many options for stuff to move from IRAM to flash to free up some space. Next step would be to integrate espusb files into bootloader. |
Would we do it where I have some code that just all lives in IRAM - then somehow, when it decides to boot, it loads the program off a different place on FLASH into IRAM and boots it? Right now, I do use library calls for a few things like memcpy, and setting up the inputs to be GPIO instead of special functs. |
I can add esp-usb to our eboot bootloader. It would be nice if there was a way to tell whether USB isn't connected at all (no cable) — and avoid loading espusb from flash in this case. Memcpy is in rom, gpio_xxx functions are also in rom if I'm not mistaken. If there are any dependencies, I'm sure we will figure this out. |
We could avoid use of interrupts and just poll for USB and/or serial? Do you use interrupts in eboot? One of the biggest questions would be how long to wait if you are connected to a host. May take a while to know if the user wants their code to run and pretend to be a mouse vs they just haven't gotten their flasher running yet. |
This is how it is done in the Arduino Leonardo board:
Is it possible to use something similar to this existing Arduino behavior as a starting point? That is, by default boot into the sketch directly without a delay unless the reason for the reboot is known to have been triggered by the Arduino flashing procedure. Here is the source code of Caterina, the bootloader used in Arduino Leonardo: https://github.com/arduino/Arduino/blob/master/hardware/arduino/avr/bootloaders/caterina/Caterina.c |
As mentioned above, because of IRAM usage it is likely that we integrate esp-idf into bootloader only, but not in the app. In this case we can assume that if the device is plugged in, it should wait for upload. If it isn't plugged in, it skips esp-idf completely and starts up as usual. |
https://github.com/cnlohr/espusb/blob/master/user/usb.c#L224 I think this may be helpful! So google can find this: This is a picture of a powered-off ESP8266 being plugged into USB. It takes a moment, but the host does talk to it and enumerates Re: CDC, etc. Even a bulk endpoint, I'm OK with, but the problem I want to avoid is I just hate it when things are COM ports, so many driver issues and the like! |
Is there still some hope that this might happen some day? |
I keep kicking the idea around, but, there's just SOOO much. I don't even know how to do the remapping of the IRAM stuff, so I'm pretty stuck at this point. Given a framework, I can totally write the USB code. |
Thanks @cnlohr, what exactly do you mean with a "framework"? |
I can get messages to/from the host via USB, and detect when it's plugged in or not, but I don't know how writing to flash works realistically, where different things would live in flash, and how to actually boot into flash. |
@igrr can you help on the above? I think this would be a tremendous benefit for the whole platform. |
FWIW, I'd love to see support for the Arduino Serial, Mouse & Keyboard libraries on ESP8266 with this also. Not sure if that's harder or easier to implement. |
No way to do serial, but things like mouse/keyboard/joystick shouldn't be hard. Really, though, I think a bootloader is the critical point! |
I agree, the bootloader and firmware updates are more important, but having USB support for Arduino applications would be very useful as well. (I do realize that the native ESP8266 serial console wouldn't be possible but an application-level serial port should be, no?) BTW: THANK YOU for this awesome work. |
If you're using application-level stuff, you really really really should be using control messages rather than treating something like serial. Control messages allow you to say "Function 5, length 50, here's my payload" which is much better than handling framing, etc. yourself. Back to discussion at hand: @igrr any ideas on how we could make this happen? Should I just make a super minimal make-usb-link thing that lives in IRAM? Or, is IRAM a bad thing when dealing with bootloaders? |
From bootloader point of view, the following functions are needed:
All functions should live in IRAM, as we have plenty of it in bootloader, and we don't use flash cache there at the moment. |
I'll work on this shortly. I am gonna see if I can do all this with as little of the SDK as possible. I am thinking the transfer thing might be a little easier to use via global messages, since, data going in and out has to be done asynchronously. I.e. "I got this data last transaction" and "when you get around to it, please send this back to the host" unless I can just have loop continuously on my side... |
I looked a few times through the code, but I guess I'm used to different style USB programming (let's say v-usb, libusb and such) and could not at all understand where to hook and how to listen for packets or send any (same for bulk). I have libs and definitions for many devices and have written lots of USB code, but I guess it was not enough :( |
@me-no-dev : The fastest way to talk to this thing is via libusb, using specifically control packets. I will admit it is a little weird, but I have a demo that runs on the USB side as well as topside to test data transfer speeds. |
@cnlohr I know what to do on the host side :) I want to be able to create different devices on the ESP, maybe even selectable which to start. |
@cnlohr maybe we can chat on gitter or something, where I can share some code with you and better explain :) |
@me-no-dev let's separate the discussion between the bootloader (i.e., flash the ESP over emulated USB) vs. "userland" (e.g., create different devices on the ESP from sketches, I have openend #24 for that). |
@probonopd it's the same exact thing ;) |
/me would like to see a YT live stream with @cnlohr @igrr and @me-no-dev :-) |
@probonopd if I am able to do what I wrote above, neither the Host side, nor the ESP side would be a problem for bootloader uploading. Maybe even exactly emulate Arduino (with resetting to bootloader and uart) as I have done it before on AVR and STM32 |
@me-no-dev Unless you know how to get CDC to run over low-speed USB, it's going to be difficult. Do you? Because that would totes be rad! Callbacks aren't quite what you'd expect, but they are close. This has to do with needing to reply on the USB bus IMMEDIATELY. It is similar to V-USB that way... The facility is sort of there, though. For example... In bootloader/main.c, line 34... it does handle control messages kind of like what you're after.
BUT!!! Keep in mind, this code has to execute /very/ fast! Code a little further on in main() is what actually handles the transaction.
I won't be free until Wednesday, late or maybe Thursday :-/ |
Most OSes will be fine with a low speed USB CDC device, even though it technically breaks standard since bulk endpoints are forbidden. I'd be more worried that you can't guarantee any real time compliance since the esp steals cycles for its wifi functions. |
bulk endpoints are not forbidden, but are rather limited in amount of data and frequency of polling, which in reality makes only bauds below 57600 or so work (do not remember what I had calculated). Bulk endpoints are used in many/most low speed usb devices :) Please correct me if I'm wrong |
It would not enumerate my serial CDC device at all. Something about "insufficient bandwidth" even when I wasn't even using the bulk endpoint. Anyone know if there's any "trick" around this? EDIT Anyway, sounds like control messages make much more efficient use of the bandwidth! |
@cnlohr what OS did you try that on? BTW did you get my email? |
@me-no-dev - Yes, I got it. Things have been insane recently, and until MAGFest is over I don't think I'll get a chance to try it. I was running on either linux 3.x or 4.4. I did notice your descriptors were different than mine. When you tested it, did it work in Windows. Is it okay for you to post the zip here so @leopck can give it a crack? |
Would be great if I could get more leads to this, since I've been stuck at this issue for a while. EDIT: I got it to enumerate, but it shows these errors. I've been getting this issue and I can't go over it. Any ideas?
|
@cnlohr about "code has to execute /very/ fast!": you should be able to NACK a couple of times and then give the answer. Maybe you can use a queue for responses and send those or NACK if none. |
Actually, Micah in a different thread on Twitter pointed out you can pre-emptively ACK as well, but, based on my traces that might not be the best move. At low-speed I think we'll have time, just not to go and do a ton of stuff. EDIT: zip was forwarded to @leopck |
I know it's a dead thread but i would be very nice if you guys still working on this. I tried to port it myself but it's too hard for the first time of doing this |
@cnlohr your README says that SDK >= 2.0 are incompatible. We (Arduino) are currently at SDK 2.2+. |
The reason originally was because the new SDKs use up precious IRAM. Espressif indicated this was something they would consider reducing in the future. I have not tried a new SDK to see if they have reduced their IRAM footprint. If they reduced it, then it will work A-ok! |
The newest SDK still uses too much RAM. Emulating USB via software is right
on the edge of the capabilities of this device. I've switched to ESP32 for
some projects, although the disparity in cost is a bit of a pain. ESP8285
is my favorite price point/size/speed microcontroller of all time, though.
…On Wed, Feb 28, 2018 at 12:44 PM, CNLohr ***@***.***> wrote:
The reason originally was because the new SDKs use up precious IRAM.
Espressif indicated this was something they would consider reducing in the
future. I have not tried a new SDK to see if they have reduced their IRAM
footprint. If they reduced it, then it will work A-ok!
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ABf2gLnj0OdxOHHGS0ghDFAiD1V5y_mLks5tZayogaJpZM4Jewxs>
.
|
It's so frustrating cause it would be so easy for them to move more stuff out of IRAM... Even an extra 250 bytes would make it so much easier. @SpiraMirabilis How much is it overflowing for you? |
@cnlohr I took a look at the code in this repo, and it looks rather alien to me. Can we discuss directly? I'm on gitter. |
I'm not really gonna have a ton of time to dig into this for a bit :( |
@cnlohr have you gotten back to this at all? A lot has changed since I last looked, including:
|
I have not examined this at all. Would be cool if someone else could take this and run. |
@cnlohr well, I've been looking for a while now for someone to do that, but I haven't found anyone who understands both usb and asm, and also has the interest and availability to pursue. |
@devyte honestly, you shouldn't need to touch any of the ASM... You WILL need to know some basic USB, especially to make apps on the host side to talk to the things. Maybe there's something I can help with. You can contact me directly if you'd like an easier back-forth support. |
Great project - I hope it can be nicely integrated with https://github.com/esp8266/Arduino (e.g., to do the uploading and "serial" monitor using just the ESP and no serial adapter).
I also cross-posted this at esp8266/Arduino#2375
The text was updated successfully, but these errors were encountered: