AI In the Workplace

On 29 June 2024, it was my pleasure to deliver a presentation as part of the ACB Convention (Virtual) on behalf of the Blind Information Technology Specialists.
BITS is an extremely worthwhile and valuable organisation promoting the use of technology as used by blind people. I highly recommend you access the BITS web site to explore what they have to offer.

While you can Download the audio presentation without Q and A here, the primary purpose of this page is to provide a text transcript of it.
This is DeafBlind awareness week and it is essential that noone is disadvantaged and should be able to have access to everything I had to say. I hope therefore the below transcript in text form is useful.

Brian Hartgen: Welcome to my presentation for ACB, Using AI in the Workplace. I'm Brian Hartgen and during the next little while, I'm going to be talking about and demonstrating something which I think has become one of the most interesting and innovative changes to technology in recent years. It is used by anyone working with mainstream technology, but as I will discuss, it has particular benefits to users of screen readers. I'm talking about the use of AI or artificial intelligence. I'm sure that AI is a term that you are familiar with, even if you haven't experienced the technology that comes from it. AI is a term which is all-embracing. It can be used to describe manipulating technology in many different areas. What I will be discussing here, however, is accessing useful information together with the processing of images so as to make them accessible to us.
I will focus upon how to use some of the tools to work with AI in the workplace. I'm using the term workplace in its loosest sense, because everything I will be describing here can be useful both at home and in educational settings too. I will be using JAWS for Windows as my screen reader, and I'll discuss what works well and where some improvement could be made. The impact of artificial intelligence related tools has been astonishing. Their impact is as profound as the introduction of the electric light bulb. I do not want to spend too much time on a discussion of AI itself as I would prefer to devote the time that I do have to demonstrating different services, and you've already had some sessions on that subject anyway. But in my view, AI gives us as blind people access to an enormous range of information which we would've had to search for in other ways.
Perhaps by reviewing a multitude of different websites or search engines, it can be used to consolidate information quickly so as to bring it to us in a more convenient form, and it can also be used to construct data for you based upon what is fed to it. As you would imagine, there is considerable controversy and varying schools of thought surrounding the use of AI. There are a number of AI search tools around. As with many resources, people will have their particular favourites. Some people will say that some services work better than others, but perhaps more importantly, AI is not always accurate. While we should keep in mind that there are inaccuracies with these tools, certainly in terms of interpreting the content of images, what we're getting back from them is significantly better than what we've ever had access to. While, for example, no one would discourage anyone from asking people to add alternative text descriptions to images, they are in no way as detailed as anything that AI is giving to us through its various services. I would far rather have what we've got now than what we had in the past.
The other point that I want to make is that within every AI-related service, prominence is given to emphasising that there may be inaccuracies. So it's not like you don't know that this could happen. If it does occur, each service has a mechanism by which you can submit feedback so as to state what is wrong. So to some extent, these services will only improve if we submit the appropriate feedback. The onus is on us, in part, to improve matters. In terms of using AI in the workplace itself, it could well be that you simply will not be able to, and that is because the IT department of your organisation may not allow you to send data out over the internet.
I suspect many organisations are still themselves figuring out how they can best utilise AI-related services, and you will need to discuss that with them. Time in this presentation is short, so I would rather now focus upon some of the things which can be achieved through various services, paying particular attention to how best to interact with them using JAWS. The services are changing every week, so I will try to give some strategies concerning things to look out for so that you can adapt what we have at present to something which you can use in the future, perhaps with your favourite AI-related tool.
I'm going to begin with Microsoft Copilot. This is available in two tiers, a free version and one you can pay for. The latter is called Copilot Pro. I have a subscription to Copilot Pro, and one of the core differences between the two different versions is the speed at which it processes requests that you submit. You'll need a Microsoft account to use Copilot. One of the reasons I like Copilot so much is because it contains good heading markup on its various pages, and also prompts which are sent directly to the screen reader automatically so you know exactly what is happening. Those prompts are quite verbose, but we can do something about some of those and I'll get to how you might go about it. Other services that I will refer to later do not include this, and the webpages concerned in those other services also contain many unlabeled graphics and controls.
You can use Copilot from within the Microsoft Edge browser, from an Office 365 application, which I will return to later, or from any web browser. If you're using a web browser, you will go to copilot.microsoft.com. I have a shortcut on my desktop to go straight to that URL whereupon I can submit a query as I'm going to do shortly. As of this moment, using Windows 11, the keystroke Windows plus C will also bring up Copilot right away, although that is under some discussion as to whether that keystroke will continue.
If it doesn't, there's nothing wrong with you using the method that I've just described, a shortcut pointing to the website, or indeed, you could create a shortcut pointing to the Windows app itself For Copilot. Bear in mind, if you use that keystroke, Windows C, it is the Microsoft Edge browser which is being used rather than perhaps your default and preferred web browser. However you choose to access Copilot, the layout of the page is very easy to work with. In terms of what to ask for when submitting a prompt, it can literally be anything you like. Let's try a couple of examples. I'm going to go to the top of the page with Control-Home.
JAWS: Copilot.
Brian Hartgen: And then type the letter E for the first edit field.
JAWS: Ask me anything, dot dot dot. Edit, blank.
Brian Hartgen: Typically, most AI tools work like this. You go there because you want to ask for something. So by and large, you're going to find an edit field close to the top of the page and you can submit your prompt. I'm going to go into forms mode.
JAWS: Forms mode on. Secure connection. Ask me anything, dot dot dot. Edit.
Brian Hartgen: By pressing enter, and I'm going to type, "Tell me about the ACB 2024 Convention." So I'm not even going to specify at this stage that it relates to the American Council of the Blind. I'm now going to press Enter and we'll listen to what happens. Now, bear in mind that I'm using Copilot Pro, as I said. I'm not going to edit the response time, so what you hear is exactly what you would ordinarily experience.
JAWS: Searching the web for ACB 2024 Convention. Stop responding button, ask me anything, dot dot dot. Edit, blank placeholder. Ask me anything. Dot dot dot. Response stopped. Received message. The 2024 American Council of the Blind ACB Conference and Convention will take place from July 5th to July 12th at the Hyatt Regency Riverfront Hotel in Jacksonville, Florida. The opening general session is scheduled for Sunday evening, July 7th, and a banquet will be held on Thursday night, July 11th. Throughout the convention, there will be day-long tours, special interest affiliate sessions, committee meetings and opportunities for collaboration with the Blinded Veterans Association, BVA.
Brian Hartgen: So you heard the response being read out automatically. I didn't have to do anything special, but of course we can review that text because as well as providing useful information, each AI-related service such as Copilot will also show you the sources from where the information derives. So you are able to go to the individual websites of those sources if you want to refer to information in more depth. I am obsessed with improving productivity and making sure that speech prompts are less verbose than perhaps we have at the moment. So let's talk through some ways that we can refine what we heard. I'm going to go into the JAWS speech history so that we can review some of the prompts that we did here. In order to move into speech history, you press Insert-Space or Caps Lock with Space, depending upon the JAWS keyboard layout you are using, followed by H for history. I've taken us to the point in the speech history where I pressed Enter to submit the prompt. We'll go through it line by line.
JAWS: Searching the web four ACB 2024 convention.
Brian Hartgen: That's fair enough.
JAWS: Stop responding button. Placeholder, response stopped.
Brian Hartgen: We've got a few phrases there. Stop responding was one of them that perhaps we didn't need to hear. Placeholder is another one. Response stopped is a third. We want to eliminate those. So what I would suggest is that if you really want to do that, you copy out those phrases, put them in the JAWS dictionary, and suppress the spoken output. In other words, you do not include a substitute prompt. So I'm going to copy this one.
JAWS: Response stopped.
Brian Hartgen: Which is response stopped.
JAWS: Selected, response stopped, copied.
Brian Hartgen: Now I'm just going to close down the speech history and go back to my browser.
JAWS: Copilot, Brave, M-Brian Hartgen.
Brian Hartgen: JAWS key with D for the JAWS dictionary.
JAWS: New dictionary dialogue. File name, edit, chrome.jdf.
Brian Hartgen: Press Enter.
JAWS: Add dot dot dot button, Alt plus A.
Brian Hartgen: Space on the add button.
JAWS: Add dictionary definition dialogue.
Brian Hartgen: Paste in the phrase that we've got here.
JAWS: Pasted, actual word. Edit response stopped.
Brian Hartgen: Tab.
JAWS: Replacement word. Edit, Alt plus R.
Brian Hartgen: Press the Space bar for nothing, because we don't want anything to be spoken here. And now I'll press Enter.
JAWS: Chrome.jdf JAWS dictionary manager.
Brian Hartgen: Alt, F4.
JAWS: Dictionary manager dialogue. Save changes to Chrome.jdf? Yes button, Alt plus Y.
Brian Hartgen: Space on the yes button.
JAWS: Copilot, Brave, M-Brian Hartgen.
Brian Hartgen: Now in a moment, we are going to have a look at another prompt, and when we execute that, hopefully we won't hear the phrase, "Response stopped," but you can apply that to any phrase. Now let's go back to what we were interested in. We heard the details of the submitted query, but we might want to review it. So let's go to the top of the page.
JAWS: Copilot.
Brian Hartgen: If you were to traverse through the headings by pressing the letter H, you'll hear the heading levels assigned to the different parts of it, and that is important because what we're going to do is to shortcut that a little bit, and we're going to use the number keys on our keyboard just above the letter keys in order to move directly to the part of the information which interests us. On a lot of services such as Google, Gemini, ChatGPT or Perplexity, which are other AI-related services, it could well be that the heading Markup isn't as good, but it certainly is a mechanism by which you can move through the page more quickly. I'll show you another method using Google Gemini in a moment that you might like to take advantage of with your alternative AI-related tool. But this is well marked up and I know items at heading level five are my questions, and responses from Copilot are at level six. I'm at the top of the page. If I were to press number six now...
JAWS: Tell me about the ACB 2024 Convention, heading level five. The 2024 American Council of the Blind ACB Conference and Convention will take place from July 5th to July 12th at the Hyatt Regency Riverfront Hotel in Jacksonville, Florida.
Brian Hartgen: That's a good example. We don't have any more prompts at the moment, but you can hear that it's taken me directly to that spot on the page, and I can review that with the arrow keys and refer to any links which may be there. There are other options here, various buttons that we can move through. I'll press the letter B.
JAWS: Like toggle button, dislike toggle button.
Brian Hartgen: If you think the response is good, it may be worth activating the like button by pressing Space bar or Enter. That influences the way that the tool works. If you dislike something, you have the opportunity of submitting feedback to determine what was inaccurate about it.
JAWS: Read aloud button.
Brian Hartgen: You can have this read aloud, if I press Space.
Speaker 3: The 2024 American Council of the Blind, ACB Conference and Convention will take place from July 5th to July 12th at the Hyatt Regency Riverfront Hotel in Jacksonville.
Brian Hartgen: So you can hear that it's more human sounding than the eloquence speech. Now we have a series of suggestions, and again, I'm pressing the letter B to move through the buttons here rather than having to arrow all the way through the page. And so what I'm doing is bypassing the intervening text that may be on there. I'm just going to the parts of the page, which are interesting. I can press Enter or Space on any one of these buttons. Each button relates to a possible follow-up question, and pressing Space or Enter on such a button will generate the answer to that question.
JAWS: "How can I register for the convention," button. "Are there any virtual sessions available," button.
Brian Hartgen: So if you didn't want to type a follow-up question yourself, then you could certainly activate any one of those buttons, but I am quite capable of sending in a follow-up question, so I'm going to do that now. Back to my edit field by pressing E.
JAWS: Ask me anything, dot dot dot, edit blank.
Brian Hartgen: Enter for forms mode,
JAWS: Forms mode on.
Brian Hartgen: Tell me about the hotel. I'm not referring any longer to the ACB 2024 Convention. It's a follow-up question. Copilot already knows the subject that we're talking about, so I don't need to allude to it. I'm going to press Enter, and remember that we shouldn't be hearing the term response stopped because we added that to the dictionary.
JAWS: Searching the web for Hyatt Regency Riverfront Hotel Jacksonville, Florida. Stop responding button. Blank placeholder. Ask me anything, dot dot dot. Receive message. Certainly, the Hyatt Regency Jacksonville Riverfront is a non-smoking hotel located adjacent to the St. Johns River. It's just a six-minute walk from shops and dining at Jacksonville Landing.
Brian Hartgen: It's going to go on now and give me details about what the hotel has to offer. You'll notice that we didn't hear, "Response stopped." We did hear, "Stop responding," because we haven't added that to the dictionary, and we heard reference to a placeholder. But if you wanted to add all your own dictionary rules there, you could. Let's go back to the top of the page.
JAWS: Copilot.
Brian Hartgen: And if I press number six.
JAWS: Tell me about the ACB 2024 Convention, heading level five. The 2024 American Council of the Blind ACB Conference and Convention with-
Brian Hartgen: That was my first prompt and corresponding answer. If I press number six again...
JAWS: Tell me about the hotel, heading level five. Certainly. The Hyatt Regency Jacksonville Riverfront is an non-smoking hotel located adjacent to the St. Johns River.
Brian Hartgen: So you can see how easy it is in order to move through the various prompts and also to get the answers that we're interested in.
Now I want to try Google Gemini. It was previously called Google Bard. The reason that I want to do this is to show you another mechanism by which you could locate important information. The response that we get from JAWS is not so good using this AI tool, and that's why I'm choosing it. Because if you use one that is not so well equipped, you can apply a good strategy, similar to the one that I'm going to show you. So I'm going to go to gemini.google.com. I have a Google account and I'm signed into it. So now I'm going to go through a similar routine, go to the top of the page.
JAWS: Gemini.
Brian Hartgen: E.
JAWS: Enter a prompt here edit, contains text.
Brian Hartgen: Go into forms mode.
JAWS: Forms mode on. Secure connection.
Brian Hartgen: We'll just use the same question. I'll type the question and press Enter.
JAWS: Gemini is typing. Gemini replied.
Brian Hartgen: It's very fast, but we don't know where the answer is, so we need to go through the headings here and find out.
JAWS: Recent, heading level one. Conversation with Gemini, heading level one. Tell me about the ACB 2024 Convention, heading Level two.
Brian Hartgen: Heading level two is where my question is, but if I arrow down, you'll see that we've got a lot of intervening text that we don't want.
JAWS: Edit button.
Brian Hartgen: There's an edit button.
JAWS: Show draughts button collapsed. Listen button.
Brian Hartgen: We can listen to it.
JAWS: Rocket, unlabeled graphic.
Brian Hartgen: There's a graphic here.
JAWS: Unlabeled graphic, the American Council of the Blind, ACB 2024 Conference and Convention is being held in the hybrid format for the third year in a row from July 5th to July.
Brian Hartgen: So you hear that we've got different information, and that is one of the interesting things about AI-related services in that the style of information is going to be different. I have to say that with Copilot as well, you can change the presentation style of the information to make it more precise or a little more relaxed and informal, and that depends upon the situation that you happen to be working with. Of course, if it was very factual information that you were wanting that you were going to use elsewhere, perhaps in your writing, you would want to change the option achieved by a series of radio buttons so as to alter the writing style. That's not what we're interested in at the moment.
You'll notice that the start of what we're interested in is not denoted by a new heading, so we perhaps need to do something about it. And one of the things that you can do is to set a place marker. When you are setting a JAWS Place marker, which is essentially a bookmark on a specific point of a page, it is worth taking some time to have a listen to the content of what is on that page, and you're going to find something that it can latch onto, which is consistent with all pages on the site. And we do have something interesting here.
JAWS: Rocket, unlabeled graphic.
Brian Hartgen: We have something here called rocket. It's a graphic a couple of lines above where we actually need to be, which is the start of the response. That's as near as we're going to get. So I've placed the cursor on that. I'm going to press Ctrl, Shift, K.
JAWS: Place marker, list dialogue. Place marker, list view. Zero items.
Brian Hartgen: Tab to add.
JAWS: Add dot dot dot button, Alt plus A.
Brian Hartgen: Space.
JAWS: Add place marker, dialogue. Name, edit, rocket, Alt plus N.
Brian Hartgen: And you'll hear that the rocket graphic is in the edit field, which is fine. You do not need to keep that title. I'm going to press Tab.
JAWS: Define for all pages on current domain, checkbox not checked. Alt plus D.
Brian Hartgen: Check the box.
JAWS: Checked.
Brian Hartgen: Tab.
JAWS: Anchor to text. Check box not checked, Alt plus A.
Brian Hartgen: Check the box
JAWS: Checked.
Brian Hartgen: We're checking this box because we want the place marker to latch on to specific text. Tab.
JAWS: Anchor text. Edit, rocket, Alt plus T.
Brian Hartgen: That's good. Tab.
JAWS: Okay button. Alt plus O.
Brian Hartgen: And press Space. Let's go to the top of the page.
JAWS: Gemini.
Brian Hartgen: And type the letter K.
JAWS: Rocket, unlabeled graphic.
Brian Hartgen: That's good. Now all we need to do is arrow down a couple of lines.
JAWS: Unlabeled graphic, the American Council of the Blind, ACB 2024 Conference and Convention is being held in the hybrid format for-
Brian Hartgen: And there you are. Not perfect, but at least you are at the start of where you need to be. Within your favourite AI service, you may well want to set up JAWS place markers at strategic points on the page. A good example as far as Gemini is concerned will be the New Topic button, which allows you to start out a new conversation. If you don't start out a new conversation, the service will think that you're asking questions about the original subject matter. So it is important to make sure that you do start out a new conversation if you're going to talk about something new.
I was talking earlier about Microsoft Copilot, and that can be used to some degree within Office applications. For the best screen reading results, of course, we use the standalone Office applications rather than those which are web-based apps. At the moment, therefore, the access to Copilot is more limited. This is particularly the case within Microsoft Outlook, although Microsoft do tell us that this will change a little bit later on.
But Microsoft Word is one place where we can use it effectively. One of the things that you might ask Copilot to do is to draught some content for you. If you're not too good at explaining things or you need something rephrasing, you might need some help, and Copilot can draught something which might be more helpful for you. For example, I'm going to draught a letter applying for the position of a JAWS script writer. At the appropriate time, I will type, "Draught a letter applying for the post of JAWS script writer." I have been writing JAWS scripts since 2001. I have been working in the access technology industry for 30 years. In terms of scripting, I have regularly carried out work for government departments and organisations so as to ensure applications are accessible. I particularly enjoy creating scripts for applications where people have deemed them to be inaccessible. I'm used to working with people with varying computing abilities. Creating documentation to explain concepts within the scripts is just as important as creating the scripts themselves.
So those are my six highlighted points, my bullet points, so to speak. I have created those and I'm going to open up Microsoft Word now. I'm in a blank document, but I could be anywhere in the document concerned. I'm going to press Alt with I now, which is the Microsoft Word command in order to move into Copilot.
JAWS: Draught with Copilot, create prompt. Zero slash 2000. Prompt edit, contains text.
Brian Hartgen: This is a very simple dialogue. There is an edit field which is in focus right now. It's waiting for anything up to 2,000 characters for our item that we want to submit, and I'm going to paste in here exactly what it is that I just read out to you.
JAWS: Pasted.
Brian Hartgen: And now let's just review that to check it.
JAWS: Top. Draught a letter applying for the post of JAWS script writer.
Brian Hartgen: The only other button of significance here is the Generate button. I'm going to press Enter on this button and I will let you hear exactly what happens. Then we will have a look at the draught itself and decide whether to discard it, regenerate it using different phrasing, or indeed to keep it
JAWS: Creating a draught, dot dot dot. Creating a draught, dot dot dot. Stop generating button. Pulling things together, dot dot dot. Working on it, dot dot dot. Draught with Copilot. Draught ready, end document. Keep it button.
Brian Hartgen: We have a series of buttons in focus that we can tab through. The focus button is Keep it, but I'm going to press Shift and F6, or you can press F6 in order to get to the draught itself.
JAWS: Leaving ribbons, edit.
Brian Hartgen: And you can tell that you are there because it says edit. Now we will go to the top of this and we will have a listen to the letter that it's come up with.
JAWS: To whom it may concern, I am writing to express my interest in the position of JAWS script writer as advertised on your website. I have been writing JAWS scripts since 2001 and I have been working in the access technology industry for 30 years. I have extensive experience in creating scripts for various applications, both mainstream and specialised to ensure their accessibility for users of JAWS. I have regularly carried out work for government departments and organisations such as the Department of Education, the National Library, and the Royal Society for the Blind. I have also provided training and support for JAWS users and script developers. One of the aspects of scripting that I enjoy the most is finding solutions for applications that are considered to be inaccessible or challenging. I have successfully scripted for complex web applications, graphical user interfaces and multimedia content. I have also developed scripts for customised software and hardware such as Braille displays, scanners, and printers.
I understand that creating documentation to explain the concepts and features of the scripts is just as important as creating the scripts themselves. I have written user manuals, technical guides, and online help files for the scripts that I have developed. I have also created audio and video tutorials and demonstrations to illustrate the functionality and benefits of the scripts. I am used to working with people with varying computing abilities from beginners to advanced users. I have excellent communication and interpersonal skills and I can adapt my style and approach to suit the needs and preferences of the users. I am also able to work independently or as part of a team, and I can manage my time and tasks efficiently and effectively. I'm confident that I have the skills, experience and passion to be a valuable member of your organisation. I would love to have the opportunity to discuss the position with you further. Please find attached my resume and portfolio of some of the scripts that I have created. Thank you for your consideration. Sincerely, your name.
Brian Hartgen: So that did quite a good job, didn't it, based upon the information that I gave it. Now of course, the more you give it, the better it will be. And the other thing to point out here is that, for example, while I may not have created scripts for those organisations that it referred to, it's just a template. It's just a guide for you in order to be able to modify, and you can send the letter with your specific information, but it certainly did phrase it in a way that I would think would be acceptable. So that's just one use of Microsoft Word and Copilot. There are other things that you can do, but this is rather a whirlwind tour of some of the services.
I would like to see the ability for you to be able to query the contents of a document. If you had a document loaded and it was an instruction manual for a portable media player, for example, I might want to ask it for how long do I need to charge the battery, or can you tell me where the volume control is located on the device. And it would do that based upon the contents of the hundreds of pages that may be found in a user guide. So that kind of thing would be very nice if you could interrogate Copilot as to the content of the document so that you can get it to summarise some of the things that might be in there. But it's very early days for Office and Copilot, and certainly the functionality, as I've said, is far better with the web-based applications rather than those that are standalone. But since it seems as though we may get access to more advanced Copilot features in respect of the standalone apps, then I'm very hopeful for what we might be able to achieve in the future.
One of the advances in recent AI technology has been the ability for various meeting platforms to summarise activity which has taken place there. Certainly, Microsoft Teams has this ability, as does the Zoom conferencing platform, which we're using now. I'm going to demonstrate how that works in Zoom. It is useful perhaps if you have missed part of a meeting or maybe you just need to have one or two points clarified. What I've done here is to take a podcast episode from ACB. It's a presentation that was given some weeks ago by Janet Dickelman about the ACB 2024 Convention, so I'm sure I'm not going to be sued for using this. I'm going to feed that into a Zoom meeting, and then once it's played out some of the content, I'm going to come back to you and I'm going to ask it some questions using the AI facility. So first of all, let's start off our meeting and I want you to listen to what it says.
JAWS: Zoom Workplace. AI Companion has been turned on.
Brian Hartgen: It says, "AI Companion has been turned on." If you want a meeting summarised, that is an important factor. In your Zoom settings or, indeed, Team settings, if you're using Microsoft Teams, you will need to make sure that the appropriate service is turned on for meetings, and you can do that within the settings of your account. If you're using Microsoft Teams within an organisation or place of work, it's very likely that the administrators or IT section of your organisation will need to give permission for that particular service. In order to ask a question, we need to tab in the meeting window until we get to the appropriate button.
JAWS: AI companion button.
Brian Hartgen: Here's the AI companion button. We're going to press Space.
JAWS: Context menu, ask meeting questions.
Brian Hartgen: Press Enter.
JAWS: Expanded, leaving menus, meeting questions. Input chat text, ask anything about this meeting. Dot dot dot, edit.
Brian Hartgen: So now I'm in the edit field and I can ask my question. I'm going to play in some snippets of some of this audio so that you can verify what Janet originally said, and then we will compare it to what AI gives us.
Artis: Tonight is Janet Dickelman, the ACB convention coordinator. So I'm going to hand it over to Janet and she can handle the call in whatever way she likes.
Janet Dickelman: All right, thank you, Artis. Good evening, everybody. Thank you for being here on this holiday weekend. I hope we have a few people listening. So what I'm going to do, I'm going to start and just give a little bit of information about convention registration. Then I'm going to talk a little bit about how the convention is set up, what's happening what days.
Brian Hartgen: So first of all, I'll ask who is talking.
JAWS: Sent. From meeting questions, Janet is talking.
Janet Dickelman: Registration for ACB members is $45, and for non-members it's $60. However, if you'd like to save $5 and become a member of ACB At-Large, you can set up your member account, wait a few hours, then go in and join ACB for $10 and then give it a little more time and then go in and do your registration at the $45 amount rather than the $60 non-member, and then you'll get to be a member, At-Large of ACB.
Brian Hartgen: How much is registration?
JAWS: Sent. From meeting questions, registration for ACB members is dollar 45, and for non-members it's dollar 60. However, if you'd like to save dollar five and become a member of ACB At-Large, you can set up your member account for dollar 10.
Janet Dickelman: There are two ways to register. You can register online or you can register by telephone. Call me at 651-428-5059, and I'll give that again at the end of the call, and I can help you set up a member account.
Brian Hartgen: I missed the phone number to call in order to register. What is that?
JAWS: Sent. From meeting questions, the phone number to call for registration is 651-428-5059.
Janet Dickelman: Reasons for registration are many. A, you will get all the Zoom links for all of the virtual and the hybrid sessions. B, you will be eligible to win door prizes, whether you are virtual or whether you are in person. You can become a convention sponsor. You get your name in the newspaper, you get a sponsor ribbon if you're in person and your name is read during general session.
Brian Hartgen: What are the benefits of registration?
JAWS: Sent. From meeting questions, the benefits of registration include receiving all of the Zoom links for all of the virtual and hybrid sessions, eligibility to win door prizes, the opportunity to become a convention sponsor and have your name in the newspaper and on the sponsor ribbon if you're in person. Your name may also be read during general session if you're in person.
Brian Hartgen: I don't know about you, but I think that's particularly impressive and it's a very good way of being able to work within a meeting, particularly if there are things that you would like to be clarified.
I will now discuss with you one of the most significant innovations of the JAWS screen reader in recent years. This is called Picture Smart. It enables you to quickly get detailed descriptions of images encountered on websites and within applications, for image files saved on your computer, and for screenshots and open windows. It is very quick and easy to use. You can put this to all kinds of purposes. To get a description of a YouTube video or live TV show, an inaccessible capture on a website, an image on social media, and far too many use cases to describe here. Microsoft have teamed up with an organisation called Be My Eyes so as to Offer an alternative Windows app with similar functionality.
But I like the JAWS Picture Smart interface because it's built right into the screen reader, so you can use it without having to instal anything additional. It's also much easier to review the text and perhaps copy it out to an alternative application, if needed. Because this is about AI in the workplace, I'm first of all going to load up a worksheet in Microsoft Excel containing a pie chart. A pie chart is a tool to display basic statistical information in a pictorial way, with each slice of the pie representing part of the data. JAWS has always been good at briefly describing the content of pie charts. However, as you are about to experience, when used in conjunction with Picture Smart, we're going to obtain much more detail. Here I am in my Excel workbook, and this worksheet is all about the sales of pies, believe it or not. Now, there is a pie chart here, and I'm going to use Picture Smart to describe this, if possible. The first thing we need to do is to make sure that the workbook is maximised, so I'll just do that now by pressing Alt-Space.
JAWS: Restore, R.
Brian Hartgen: Followed by X.
JAWS: Leaving menus. Nine, B5.
Brian Hartgen: And now I'm going to press JAWS key with Space. That's the Caps Lock key or the Insert key, depending upon the JAWS layout that you're using, followed by P for Picture Smart.
JAWS: Picture Smart.
Brian Hartgen: And then we're going to use Shift with W. So ordinarily, you could press the letter W in order to capture the entire window and submit it for processing, but if you press Shift with W, this gives you much more information via two distinctly different AI services. One is called Claude, and one is called ChatGPT. I like this particular feature, but at the moment there is a limit on it. I believe that you can only get access to this more detailed description via these two services about 50 times over a 24-hour period. That may well change. If you just press the letter W by itself, that's going to give you brief details. You can get the more thorough detailed information afterwards if you want. There is a link in the results viewer window in order to obtain it, but let's go for the whole kit and caboodle, shall we? I'll press Shift with W.
JAWS: Picture Smart is in progress. Heading level two, Claude.
Brian Hartgen: That's the first service, Claude, but I prefer ChatGPT myself, so I'm going to press H to go to the next heading.
JAWS: ChatGPT, heading level two.
Brian Hartgen: And we'll read the description here.
JAWS: This image is a screenshot of an Excel workbook. The workbook is currently displaying a worksheet with the title, "Exploding pie chart sample." Blank. The upper part of the worksheet contains a small table labelled pie sales with the following data. In cell B2, strawberry with a value of six. In cell C2. In cell B3, peach with a value of five. In cell C3. In cell B4, mincemeat with a value of seven. In cell C4. In cell B5, lemon with a value of three. In cell C5. In cell B6, blueberry with a value of three. In cell C6, blank.
To the left of the data table, there is a 3D exploding pie chart visualising the pie sales data. Each segment of the pie chart corresponds to a different pie type as distinct colours. Strawberry is coloured red. Peach is coloured orange, although this specific colour isn't very clear in the screenshot. Mincemeat is coloured purple. Lemon is coloured green. Blueberry is coloured blue. Blank. The chart is exploding, which means each segment is slightly separated from the others to emphasise them. The rest of the Excel sheet shows the typical Excel interface, including the menu options like file, home, insert.
Brian Hartgen: Okay, I'm not interested in that specifically, but we want more details about the pie chart itself. I'm going to press tab.
JAWS: Ask ChatGPT questions about this image link.
Brian Hartgen: We can ask questions specifically related to the image. So yes, we've got some detail about the pie chart, but we want a little bit more. Press enter.
JAWS: Type a question, edit.
Brian Hartgen: We're in the edit field. I'll press Enter again to go into forms mode.
JAWS: Type a question, edit.
Brian Hartgen: And I'm going to type, "Describe the pie chart in detail."
JAWS: Describe the pie chart in detail. Certainly. The pie chart shown is a 3D exploding pie chart, which means that some or all slices of the pie are slightly separated from each other, creating a visual effect that emphasises each segment. Here are the detailed aspects of the pie chart. One, star star, title and label. Star star. The pie chart does not have an explicit title displayed within it, but it does have a legend that associates each colour with a pie type. Two, star star segments and colour, star, star, star-
Brian Hartgen: You'll notice that JAWS is saying, "Star, star, star" quite a lot. This is coming from ChatGPT itself, and I can understand why they are separating out these different segments with asterisk characters, but it can be quite irritating. So let's do something about solving that problem. So with that results viewer window open, we're going to press JAWS key with V in order to go into quick settings.
JAWS: Quick settings, results viewer dialogue, search box edit, Control plus E.
Brian Hartgen: And I'm going to type, "Punctuation."
JAWS: Search box edit. Punctuation, punctuation. One of one search results.
Brian Hartgen: Arrow down.
JAWS: One search results, list box. Punctuation, some, editing options.
Brian Hartgen: Currently, for the results viewer, punctuation level is set to some, which is probably the default setting that I have. I'm going to press the Space bar.
JAWS: Most, three or four.
Brian Hartgen: Again.
JAWS: All, four or four.
Brian Hartgen: And again.
JAWS: None. One of four.
Brian Hartgen: And press Enter.
JAWS: Cleared. Tree View, punctuation none. Three of four editing options.
Brian Hartgen: Press Enter again.
JAWS: Describe the pie chart in detail, heading level two.
Brian Hartgen: And now let's continue to review what we've got.
JAWS: Two segments and colours. Strawberry colour, red, value six. Peach colour, orange. Though this specific shade is not very clear in the image, it is distinguishable from other colours. Value five. Mincemeat. Colour purple. Value, seven. Lemon. Colour green. Value three. Blueberry. Colour blue. Value three. Blank. Three, exploded effect. Each segment slice of the pie chart is separated from its neighbouring segments. This separation makes each slice stand out and highlights the differences in size between the slices.
Blank. Four, 3D effect. The pie chart is rendered in a 3D view, giving it a slight depth perspective. This makes the slices appear as though they are lifted off the page. Blank. Five, legend. The legend is situated to the right of the pie chart. It clearly associates each colour with the name of the corresponding pie type. Light blue square with blueberry, purple square with mincemeat, red square with strawberry, orange square with peach, green square with lemon. Blank. Six, proportions. The size of each slice is proportional to its value when compared to the whole pie. For example, the mincemeat slice is the largest since it has the highest value, seven, and both lemon and blueberry slices are the smallest, each with a value of three. Blank. Visually, the chart provides a quick and effective way to compare the sales data for the different types of pies, making it easy to see which type has the highest and lowest sales. The exploded effect further emphasises each segment aiding in distinguishing them.
Brian Hartgen: There is no way that JAWS could have provided that level of detail in the past. Now, while I was able to obtain good information, particularly in relation to the pie chart, it would've been even better if we could ask questions beforehand. So having loaded up the Excel workbook as I did a few minutes ago, I know that the worksheet contains a pie chart. I don't need to be told because I've loaded it before. Rather than having to scan it first and obtain the information that I do not need, such as the fact that we are looking at an Excel worksheet, it would've been good if I could have shortcut that and said to Picture Smart immediately, "Describe the pie chart only."
So I was able to do that, but only after the initial scan. It would've been wonderful if I could have done that in advance, but hopefully in time we will get to that. I'm sure it won't be too far in the future before we can do that sort of thing. This is becoming more and more sophisticated, and Vispero really should be congratulated for implementing that. Let's now get JAWS to identify a graphic as found on a website, such as an image on social media. I'm now on my favourite social media platform, X, formerly Twitter, and I've got a post here containing an image.
JAWS: Yesterday's Britain, a better Britain. Wonderful British people in the seventies queuing to buy an ice cream, soft ice cream.
Brian Hartgen: So the image is attached to this particular post. I am going to make sure that I'm focused on the image.
JAWS: Link graphic image.
Brian Hartgen: There it is. And I'm going to use a slightly different command this time. When you're on a website and you want to examine a particular image, you need to analyse the specific control that the cursor is on, and the command for that is JAWS key with Space, P.
JAWS: Picture Smart.
Brian Hartgen: Shift with C.
JAWS: Picture Smart is in progress. Heading level two, Claude.
Brian Hartgen: And we'll press H again.
JAWS: ChatGPT, heading level two.
Brian Hartgen: And see what it says.
JAWS: The image shows a group of people waiting in line at an ice cream van. The van is white and has text on the side that reads, "Here it is, Mr. Whippy." The people in line are of various ages, including both adults and children, and they are dressed in casual clothing. They are standing on the grassy area with a large tent in the background, which suggests that the scene might be taking place at a fair, festival or another outdoor event. The sky is clear and it appears to be a bright sunny day. The image has an nostalgic vintage feel to it, likely from the mid-20th century.
Brian Hartgen: That's a lovely description of that particular image. I'll just press Escape to come out of here. Next, I want to show scanning a file that you may have on your computer. There are a few ways of being able to do this, but one of them is to focus on the file that you may have using file explorer.
JAWS: Kate's picture dot jpg, 76 of 171.
Brian Hartgen: Here, I have an image, and this time I'm going to press JAWS Key with Space, P.
JAWS: Picture Smart.
Brian Hartgen: Shift with F. F is for file. And just to emphasise again that if you want brief descriptions, you don't need to add the shift key, and that may well be the case in the majority of instances, but I'll press Shift-F here.
JAWS: Picture Smart is in progress. Heading level two, Claude.
Brian Hartgen: H again.
JAWS: ChatGPT, heading Level two. The image consists of two main parts, blank. One, left side. A woman with long wavy blonde hair sits or stands to the left. She has an expressive face, looking somewhere to the left of the frame. The woman is dressed in a floral patterned black blouse that features intricate designs and shades of blue, green, and pink. She is leaning on or near a piano as the keys can be seen in the foreground. The background is slightly blurred, allowing the focus to remain on her and the piano. Blank. Two, right side inside image. There is a smaller black and white inside image overlaid on the right side. The inside image depicts a historic wartime scene, likely from World War II, showing soldiers in uniform. They appear to be disembarking from a landing craft onto a beach, possibly during the D-Day invasion. Link, ask ChatGPT questions about this image.
Brian Hartgen: Let's ask a question. I'll press Enter.
JAWS: Type a question, edit.
Brian Hartgen: Can you describe the piano?
JAWS: Can you describe the piano? The piano in the image is a grand piano, identifiable by its shape and the visible keys. Here are some details. Colour of the visible part of the piano is dark, likely black. Keys. The piano keys are white and black, typical of a standard keyboard layout. Logo. There is a partially visible logo on the front of the piano, which appears to be the Yamaha brand. Lighting. The lighting creates a warm glow on the piano, adding to the elegant and classical atmosphere of the image. The piano serves as an elegant backdrop to the woman, emphasising a musical theme.
Brian Hartgen: I want to end this presentation, which I hope you've enjoyed, by taking that Picture Smart in one other direction. Many of us in the workplace use web-based apps, don't we? Applications that are hosted in a browser or, indeed, just general websites, but some of the controls in such applications such as buttons and graphical images may not be correctly labelled. One of the things that you could do is to use Picture Smart in order to identify what those unlabeled graphics are, and then use the JAWS custom label feature to make the overall experience a lot easier for you to use in the future. I've got a webpage here which has two inaccessible links.
JAWS: Customer feedback.
Brian Hartgen: That's the title of the page, and if we arrow down...
JAWS: Image map, link graphic, tab two.
Brian Hartgen: That's the first one, and then if I arrow down again, I get another inaccessible link.
JAWS: Image map, link graphic, tab three.
Brian Hartgen: I need to label these up so that JAWS speaks them correctly. This is the kind of thing that you might encounter. I'm going to arrow back up again.
JAWS: Image map, link graphic, tab two.
Brian Hartgen: JAWS key, Space, P.
JAWS: Picture Smart.
Brian Hartgen: C.
JAWS: Picture Smart is in progress. The image displays the word, "Products" written in black text. The font appears to be slightly blurred or pixelated, making the edges of the letters less sharp. The background is white.
Brian Hartgen: So it's called Products. Let's escape.
JAWS: Image map, link graphic, tab two.
Brian Hartgen: Because we're still focused on the link, I'm going to press Control, JAWS key, Tab to create a custom label.
JAWS: Custom label dialogue.
Brian Hartgen: And this is called, "Products." I'll press Enter, arrow down.
JAWS: Image map, link graphic, tab three.
Brian Hartgen: Arrow up.
JAWS: Image map, link graphic, products.
Brian Hartgen: Perfect. Let's do the other one.
JAWS: Image map, link graphic, tab three.
Brian Hartgen: JAWS Key, Space, P.
JAWS: Picture Smart.
Brian Hartgen: C.
JAWS: Picture Smart is in progress. The image displays the word, "Shop" written and clear.
Brian Hartgen: That will do. Escape.
JAWS: Image map, link graphic, tab-
Brian Hartgen: Control, JAWS key, Tab.
JAWS: Custom label dialogue.
Brian Hartgen: Shop, Enter, and now we've got...
JAWS: Image map, link graphic, products. Image map, link graphic, shop.
Brian Hartgen: There you are. That's just one way of being able to use Picture Smart, the AI capabilities in conjunction with custom labels. I've not been able to really scratch the surface in terms of what AI tools can do for you, especially in the workplace, but I hope I've given you a flavour as to what is possible. I'd like to thank you for listening to my presentation today, and I hope you've got something from it. I'd also particularly like to thank the BITS Committee of ACB for inviting me to deliver the presentation for you this morning. If you would like to ask any questions, I'm sure that there will be an opportunity now for you to do so.