Improving Results Obtained from AI Assisted Development
In order to obtain quality results from AI assisted development, you need to carefully consider the size of the tasks you are asking the AI to perform, the clarity of your instructions to the AI, and whether or not you are providing enough context for it to be able to perform the requested task. If tasks are too large or poorly defined, or if you are providing the AI too little contextual information, then you will receive results that are either incorrect or require too much manual effort to work with.
For AI code suggestion, your request is essentially defined by what precedes the current editor insertion point. Adding a comment or the first part of the code you want helps the AI to understand what you expect it to write. Importing modules you expect the AI to use is another way to increase the chances that you will get what you are looking for.
For AI refactoring, description-driven code development, and AI chat, both the code in and around your selected refactoring target or focus and your written instructions are important. Be sure that your request is very specific and complete. The AI will understand more than you might expect. For example, you can ask it to "add documentation strings" but may get results closer to what you're looking for if you instead ask it to "add brief documentation strings but without documenting arguments" or "add documentation written in French".
However, asking the AI to refactor, write, or otherwise process too much code at once tends to exceed the computational limits of generative AI, and won't produce good results. Focusing on a single function, method, or class tends to work well. Trying to refactor a large file all at once often does not work well. You will need to experiment to get a sense for these limits. What works is likely to change over time, as generative AI continues to advance.
For AI code suggestion, refactoring, and chat, you will also need to think about whether the AI might need additional context that is not found in the current editor file, as described in detail in AI Request Context.
AI assistants share many characteristics of human software developers: Given an overly large task with too little time to complete it, in the case of AI due to limits on computational complexity, they become overwhelmed and give up before coming up with a good result. A poorly specified task may cause them to go off on a tangent and do the wrong work. Too much irrelevant context can similarly confuse them, wasting their time and attention.
As also for humans, work created by an AI will vary somewhat with each invocation, even if the context and request are exactly the same. If you don't like the results that you've obtained, you may sometimes be able to get what you want just by invoking the same request again. In other cases, you may be better off asking the AI to correct specific mistakes or make further enhancements in subsequent requests, rather than expecting to write all the code with a single request.
Revision Control
We strongly recommend using a revision control system so that you can review and adjust code suggested by your AI provider, before it is accepted into your code base. Wing's AI suggest, refactor, and description-driven development features will replace code in your files. If you save them to disk without a revision control system and quit Wing, then you will lose the ability to restore your code to the state it was in before your AI request was made.
If you are not using revision control and are concerned about the AI changing your code, you can use AI Chat to ask the AI to write code and then manually copy and paste results into your code base. AI Chat only outputs code to the chat area and never makes any changes to editors.
Performance
AI supported development is compute intensive. Requests often take 5 or 10 seconds to complete, and sometimes much longer than that. However for well-designed requests, the amount of work being done by the AI is considerable, and it often produces results that would take much longer to implement manually.
Be sure to also read Managing Cost so that you understand the approximate costs involved with your AI assisted development.
Language Model
The default configuration in Wing uses the latest (as of release) OpenAI's GPT 4 Turbo language model. The GPT 4 Turbo models are currently the only ones that are likely to produce good results. The reason for this is that this model has a much larger "Content Window" (128K tokens) than any pre-existing model, which allows for much more context to be provided with requests. Wing depends on this ability, particularly when Auto-Context is enabled.
For now, OpenAI is imposing fairly low token rate limits and the model is somewhat expensive to use. Because of this you may need to spend enough to obtain at least a Tier 2 account, and more likely will need to go to Tier 4 for any real production use of AI assisted development in Wing. This requires an initial investment of around US $50 into your OpenAI account, as a credit which is then spent slowly as you issue AI requests.
This is likely to change over time, and other models may appear with similarly large context windows and lower pricing.
In our tests, we did not receive good results with any other OpenAI models, including GPT 4. If you try to experiment with older models, you will probably need to disable Auto-Context and may need to reduce the Max Local Context configured on the Config: Request Context page of the AI tool, so that requests sent to the model are smaller. See AI Request Context for details.
AI Provider Account Resources
Wing tries to remove all resources it creates in your AI provider account, whenever you change projects or quit Wing. If a network outtage or other problem prevents Wing from removing the resources, it will try again later the next time it is launched, quit, or when a new project is opened. This continues until Wing can eventually remove the resources.
In the event that resources are still not being removed properly from your AI provider account, you can manually remove them as follows:
- Select Edit > Command by Name
- In the entry area that appears at the bottom of Wing's window, enter the command internal-ai-delete-resources and press Enter
This will remove all assistants and files that Wing has created in your AI provider account, even if they are currently in use. You'll need to restart instances of Wing that are using those deleted resources before their AI chat will work again.
